2024-09-10T22:35:03.942 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T22:35:03.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:35:04.059 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899748' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJgLwL2BXJLaO2vzK0EdLxomDrLlAFtnM9v3X/i2JTRbw2l64rqfUrRgERPay4q65oVkMxSyUTRrpyWiPrJ4gQ= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKbQDdxyPhSJ1Y/Op9j+3ZOjlv1mi0xxq9qhb7d2ncxK6W9Edd0+/q1GdyFd+NnlR8Uzc2C1RwaplmC4dWr4o6A= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/GLoR8VJ0tZQ3DxnxlNAfYgKwgh7ZR7hdSXpKnZJtZmVhsXsJUcOds8tzA1bQOufTFtLPo1Hb22uIdVFdhaSE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T22:35:04.060 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T22:35:04.060 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T22:35:04.061 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T22:35:04.062 INFO:teuthology.task.internal:Checking packages... 2024-09-10T22:35:04.136 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T22:35:04.136 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T22:35:04.136 INFO:teuthology.packaging:ref: None 2024-09-10T22:35:04.136 INFO:teuthology.packaging:tag: None 2024-09-10T22:35:04.136 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:35:04.136 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:35:04.136 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:35:04.281 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T22:35:04.283 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T22:35:04.290 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T22:35:04.290 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T22:35:04.471 INFO:teuthology.task.internal:Saving configuration 2024-09-10T22:35:04.491 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T22:35:04.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T22:35:04.667 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:29:59.219519', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJgLwL2BXJLaO2vzK0EdLxomDrLlAFtnM9v3X/i2JTRbw2l64rqfUrRgERPay4q65oVkMxSyUTRrpyWiPrJ4gQ='} 2024-09-10T22:35:04.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:29:59.217566', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKbQDdxyPhSJ1Y/Op9j+3ZOjlv1mi0xxq9qhb7d2ncxK6W9Edd0+/q1GdyFd+NnlR8Uzc2C1RwaplmC4dWr4o6A='} 2024-09-10T22:35:04.722 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:29:59.221270', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/GLoR8VJ0tZQ3DxnxlNAfYgKwgh7ZR7hdSXpKnZJtZmVhsXsJUcOds8tzA1bQOufTFtLPo1Hb22uIdVFdhaSE='} 2024-09-10T22:35:04.722 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T22:35:04.763 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T22:35:04.763 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T22:35:04.763 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T22:35:04.764 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T22:35:05.332 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f14d2304dc0>, signals=[15]) 2024-09-10T22:35:05.333 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T22:35:05.673 INFO:teuthology.task.internal:Opening connections... 2024-09-10T22:35:05.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:35:05.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:35:05.762 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-10T22:35:05.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:35:05.844 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T22:35:05.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:35:05.930 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T22:35:05.937 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-09-10T22:35:05.955 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-09-10T22:35:05.955 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-09-10T22:35:06.012 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-09-10T22:35:06.012 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-09-10T22:35:06.012 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:35:06.013 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:35:06.014 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:35:06.014 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:35:06.014 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-09-10T22:35:06.045 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-10T22:35:06.063 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-10T22:35:06.063 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:35:06.122 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:35:06.123 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:35:06.123 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:35:06.123 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-10T22:35:06.154 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-10T22:35:06.172 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-10T22:35:06.172 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:35:06.230 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:35:06.231 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:35:06.231 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-10T22:35:06.262 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T22:35:06.271 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T22:35:06.370 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T22:35:06.371 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:35:06.373 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:35:06.375 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:35:06.392 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T22:35:06.502 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T22:35:06.502 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:35:06.505 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:35:06.507 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:35:06.598 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T22:35:06.699 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T22:35:06.784 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T22:35:06.785 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-10T22:35:06.788 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:35:06.789 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:35:06.789 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:35:06.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:06.807 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.807 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T22:35:06.807 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-10T22:35:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:06.809 DEBUG:teuthology.task.kernel:sha1 for mon.c is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.809 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T22:35:06.809 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-10T22:35:06.809 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.810 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T22:35:06.810 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-10T22:35:06.864 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:35:06.864 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.865 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T22:35:06.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.877 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:35:06.877 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:35:06.878 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.878 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T22:35:06.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.879 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:06.879 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T22:35:06.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:07.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum upgrade -y linux-firmware 2024-09-10T22:35:07.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum upgrade -y linux-firmware 2024-09-10T22:35:07.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2024-09-10T22:35:09.267 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-10T22:35:09.638 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.2 MB 00:00 2024-09-10T22:35:10.269 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-09-10T22:35:13.842 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-10T22:35:18.405 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.5 MB 00:04 2024-09-10T22:35:23.276 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 111 kB/s | 19 kB 00:00 2024-09-10T22:35:23.848 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-10T22:35:24.613 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-10T22:35:25.640 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-10T22:35:27.127 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T22:35:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-10T22:35:30.010 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T22:35:32.116 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-10T22:35:32.117 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-10T22:35:32.117 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-10T22:35:32.213 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T22:35:32.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:32.375 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T22:35:32.375 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T22:35:32.378 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi092.front.sepia.ceph.com to complete... 2024-09-10T22:35:32.733 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T22:35:34.921 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-10T22:35:34.922 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-10T22:35:34.923 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-10T22:35:35.033 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T22:35:35.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:35.185 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T22:35:35.186 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T22:35:35.188 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2024-09-10T22:35:40.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10 22:35:40 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T22:35:40.470 INFO:teuthology.orchestra.run.smithi092.stderr:FINISHED --2024-09-10 22:35:40-- 2024-09-10T22:35:40.470 INFO:teuthology.orchestra.run.smithi092.stderr:Total wall clock time: 8.1s 2024-09-10T22:35:40.470 INFO:teuthology.orchestra.run.smithi092.stderr:Downloaded: 1 files, 343M in 7.8s (44.2 MB/s) 2024-09-10T22:35:40.473 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T22:35:40.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T22:35:41.350 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 857 kB/s | 6.5 MB 00:07 2024-09-10T22:35:42.629 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying... ######################################## 2024-09-10T22:35:42.854 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing... ######################################## 2024-09-10T22:35:42.866 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 19 kB 00:00 2024-09-10T22:35:42.975 INFO:teuthology.orchestra.run.smithi092.stdout:Updating / installing... 2024-09-10T22:35:44.446 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-10T22:35:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10 22:35:44 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T22:35:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2024-09-10 22:35:44-- 2024-09-10T22:35:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 9.4s 2024-09-10T22:35:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 343M in 9.0s (38.2 MB/s) 2024-09-10T22:35:44.606 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T22:35:44.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T22:35:46.809 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2024-09-10T22:35:46.927 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2024-09-10T22:35:47.030 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2024-09-10T22:35:49.993 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-10T22:35:51.008 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T22:35:52.114 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-10T22:35:52.115 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-10T22:35:52.115 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-10T22:35:52.212 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.c... 2024-09-10T22:35:52.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T22:35:52.363 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T22:35:52.364 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T22:35:52.366 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi145.front.sepia.ceph.com to complete... 2024-09-10T22:35:55.351 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T22:35:58.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10 22:35:58 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T22:35:58.711 INFO:teuthology.orchestra.run.smithi145.stderr:FINISHED --2024-09-10 22:35:58-- 2024-09-10T22:35:58.711 INFO:teuthology.orchestra.run.smithi145.stderr:Total wall clock time: 6.3s 2024-09-10T22:35:58.712 INFO:teuthology.orchestra.run.smithi145.stderr:Downloaded: 1 files, 343M in 6.0s (56.9 MB/s) 2024-09-10T22:35:58.718 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.c... 2024-09-10T22:35:58.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T22:36:00.835 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying... ######################################## 2024-09-10T22:36:01.082 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing... ######################################## 2024-09-10T22:36:01.184 INFO:teuthology.orchestra.run.smithi145.stdout:Updating / installing... 2024-09-10T22:36:09.180 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T22:36:31.046 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T22:36:31.047 DEBUG:teuthology.orchestra.run.smithi092:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T22:36:31.138 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T22:36:31.139 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T22:36:31.140 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T22:36:31.141 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T22:36:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T22:36:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T22:36:31.144 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T22:36:31.145 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T22:36:31.146 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T22:36:31.147 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T22:36:31.148 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T22:36:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T22:36:31.150 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T22:36:31.151 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T22:36:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T22:36:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T22:36:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T22:36:31.155 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T22:36:31.156 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T22:36:31.157 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T22:36:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T22:36:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T22:36:31.160 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T22:36:31.161 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T22:36:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T22:36:31.163 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T22:36:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T22:36:31.165 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T22:36:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T22:36:31.167 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T22:36:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T22:36:31.169 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T22:36:31.170 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T22:36:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T22:36:31.172 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T22:36:31.173 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T22:36:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T22:36:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T22:36:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T22:36:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T22:36:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T22:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T22:36:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T22:36:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T22:36:31.182 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T22:36:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T22:36:31.184 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T22:36:31.185 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T22:36:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T22:36:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T22:36:31.188 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T22:36:31.189 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T22:36:31.190 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T22:36:31.191 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T22:36:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T22:36:31.193 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T22:36:31.194 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T22:36:31.195 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T22:36:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T22:36:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T22:36:31.198 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T22:36:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T22:36:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T22:36:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T22:36:31.202 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T22:36:31.203 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T22:36:31.204 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T22:36:31.205 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T22:36:31.206 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T22:36:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T22:36:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T22:36:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T22:36:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T22:36:31.211 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T22:36:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T22:36:31.213 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T22:36:31.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T22:36:31.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T22:36:31.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T22:36:31.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T22:36:31.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T22:36:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T22:36:31.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T22:36:31.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T22:36:31.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T22:36:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T22:36:31.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T22:36:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T22:36:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T22:36:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T22:36:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T22:36:31.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T22:36:31.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T22:36:31.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T22:36:31.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T22:36:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T22:36:31.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T22:36:31.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T22:36:31.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T22:36:31.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T22:36:31.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T22:36:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T22:36:31.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T22:36:31.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T22:36:31.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T22:36:31.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T22:36:31.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T22:36:31.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T22:36:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T22:36:31.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T22:36:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T22:36:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T22:36:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T22:36:31.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T22:36:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T22:36:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T22:36:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T22:36:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T22:36:31.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T22:36:31.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T22:36:31.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T22:36:31.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T22:36:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T22:36:31.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T22:36:31.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T22:36:31.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T22:36:31.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T22:36:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T22:36:31.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T22:36:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T22:36:31.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T22:36:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T22:36:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T22:36:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T22:36:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T22:36:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T22:36:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T22:36:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T22:36:31.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T22:36:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T22:36:31.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T22:36:31.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T22:36:31.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T22:36:31.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T22:36:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T22:36:31.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T22:36:31.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:31.289 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.289 DEBUG:teuthology.orchestra.run.smithi092:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout:for file in vmlinuz System.map config; do 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: fi 2024-09-10T22:36:31.312 INFO:teuthology.orchestra.run.smithi092.stdout:done 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2024-09-10T22:36:31.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-10T22:36:31.387 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-10T22:36:31.387 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-10T22:36:31.387 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:36:31.388 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:36:31.389 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:36:31.390 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:36:31.390 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:31.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:36:31.954 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-10T22:36:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:36:33.314 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-10T22:36:33.316 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-10T22:36:33.331 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.nwOkV5XRvf 2024-09-10T22:36:33.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nwOkV5XRvf 2024-09-10T22:36:33.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.nwOkV5XRvf 2024-09-10T22:36:33.554 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.nwOkV5XRvf is 6KB 2024-09-10T22:36:33.602 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.nwOkV5XRvf 2024-09-10T22:36:33.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:36:33.679 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:36:33.679 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:36:33.680 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:36:33.680 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T22:36:33.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:33.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-10T22:36:34.865 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T22:36:34.865 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:34.956 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T22:36:34.957 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T22:36:34.958 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T22:36:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T22:36:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T22:36:34.961 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T22:36:34.962 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T22:36:34.963 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T22:36:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T22:36:34.965 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T22:36:34.966 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T22:36:34.967 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T22:36:34.968 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T22:36:34.969 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T22:36:34.970 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T22:36:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T22:36:34.972 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T22:36:34.973 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T22:36:34.974 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T22:36:34.975 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T22:36:34.976 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T22:36:34.977 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T22:36:34.978 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T22:36:34.979 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T22:36:34.980 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T22:36:34.981 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T22:36:34.982 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T22:36:34.983 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T22:36:34.984 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T22:36:34.985 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T22:36:34.986 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T22:36:34.987 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T22:36:34.988 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T22:36:34.989 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T22:36:34.990 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T22:36:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T22:36:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T22:36:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T22:36:34.994 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T22:36:34.995 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T22:36:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T22:36:34.997 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T22:36:34.998 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T22:36:34.999 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T22:36:35.000 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T22:36:35.001 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T22:36:35.002 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T22:36:35.003 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T22:36:35.004 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T22:36:35.005 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T22:36:35.006 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T22:36:35.007 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T22:36:35.008 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T22:36:35.009 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T22:36:35.010 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T22:36:35.011 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T22:36:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T22:36:35.013 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T22:36:35.014 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T22:36:35.015 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T22:36:35.016 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T22:36:35.017 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T22:36:35.018 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T22:36:35.019 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T22:36:35.020 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T22:36:35.021 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T22:36:35.022 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T22:36:35.023 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T22:36:35.024 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T22:36:35.025 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T22:36:35.026 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T22:36:35.027 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T22:36:35.028 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T22:36:35.029 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T22:36:35.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T22:36:35.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T22:36:35.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T22:36:35.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T22:36:35.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T22:36:35.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T22:36:35.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T22:36:35.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T22:36:35.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T22:36:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T22:36:35.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T22:36:35.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T22:36:35.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T22:36:35.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T22:36:35.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T22:36:35.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T22:36:35.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T22:36:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T22:36:35.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T22:36:35.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T22:36:35.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T22:36:35.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T22:36:35.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T22:36:35.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T22:36:35.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T22:36:35.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T22:36:35.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T22:36:35.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T22:36:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T22:36:35.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T22:36:35.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T22:36:35.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T22:36:35.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T22:36:35.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T22:36:35.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T22:36:35.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T22:36:35.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T22:36:35.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T22:36:35.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T22:36:35.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T22:36:35.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T22:36:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T22:36:35.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T22:36:35.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T22:36:35.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T22:36:35.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T22:36:35.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T22:36:35.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T22:36:35.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T22:36:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T22:36:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T22:36:35.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T22:36:35.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T22:36:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T22:36:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T22:36:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T22:36:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T22:36:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T22:36:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T22:36:35.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T22:36:35.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T22:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T22:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T22:36:35.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T22:36:35.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T22:36:35.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T22:36:35.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T22:36:35.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T22:36:35.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T22:36:35.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T22:36:35.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T22:36:35.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T22:36:35.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T22:36:35.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T22:36:35.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T22:36:35.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T22:36:35.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:35.110 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:35.110 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout:for file in vmlinuz System.map config; do 2024-09-10T22:36:35.134 INFO:teuthology.orchestra.run.smithi067.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: fi 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T22:36:35.135 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:35.136 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:35.136 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-09-10T22:36:35.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-09-10T22:36:35.210 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 82.el9 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8274670 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:36:35.211 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:36:35.212 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:36:35.213 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:36:35.213 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:35.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:36:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-09-10T22:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:36:37.136 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-09-10T22:36:37.138 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-09-10T22:36:37.153 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.ObIsHxSaGo 2024-09-10T22:36:37.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ObIsHxSaGo 2024-09-10T22:36:37.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.ObIsHxSaGo 2024-09-10T22:36:37.358 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.ObIsHxSaGo is 6KB 2024-09-10T22:36:37.409 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.ObIsHxSaGo 2024-09-10T22:36:37.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:36:37.487 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:36:37.487 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:36:37.487 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:36:37.487 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T22:36:37.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:37.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-09-10T22:36:49.523 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T22:36:49.524 DEBUG:teuthology.orchestra.run.smithi145:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:49.603 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T22:36:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T22:36:49.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T22:36:49.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T22:36:49.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T22:36:49.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T22:36:49.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T22:36:49.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T22:36:49.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T22:36:49.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T22:36:49.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T22:36:49.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T22:36:49.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T22:36:49.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T22:36:49.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T22:36:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T22:36:49.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T22:36:49.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T22:36:49.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T22:36:49.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T22:36:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T22:36:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T22:36:49.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T22:36:49.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T22:36:49.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T22:36:49.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T22:36:49.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T22:36:49.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T22:36:49.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T22:36:49.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T22:36:49.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T22:36:49.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T22:36:49.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T22:36:49.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T22:36:49.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T22:36:49.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T22:36:49.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T22:36:49.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T22:36:49.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T22:36:49.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T22:36:49.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T22:36:49.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T22:36:49.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T22:36:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T22:36:49.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T22:36:49.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T22:36:49.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T22:36:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T22:36:49.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T22:36:49.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T22:36:49.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T22:36:49.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T22:36:49.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T22:36:49.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T22:36:49.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T22:36:49.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T22:36:49.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T22:36:49.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T22:36:49.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T22:36:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T22:36:49.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T22:36:49.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T22:36:49.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T22:36:49.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T22:36:49.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T22:36:49.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T22:36:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T22:36:49.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T22:36:49.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T22:36:49.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T22:36:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T22:36:49.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T22:36:49.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T22:36:49.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T22:36:49.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T22:36:49.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T22:36:49.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T22:36:49.680 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T22:36:49.681 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T22:36:49.682 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T22:36:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T22:36:49.684 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T22:36:49.685 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T22:36:49.686 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T22:36:49.687 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T22:36:49.688 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T22:36:49.689 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T22:36:49.690 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T22:36:49.691 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T22:36:49.692 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T22:36:49.693 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T22:36:49.694 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T22:36:49.695 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T22:36:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T22:36:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T22:36:49.698 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T22:36:49.699 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T22:36:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T22:36:49.701 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T22:36:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T22:36:49.703 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T22:36:49.704 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T22:36:49.705 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T22:36:49.706 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T22:36:49.707 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T22:36:49.708 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T22:36:49.709 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T22:36:49.710 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T22:36:49.711 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T22:36:49.712 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T22:36:49.712 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T22:36:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T22:36:49.714 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T22:36:49.715 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T22:36:49.716 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T22:36:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T22:36:49.718 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T22:36:49.719 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T22:36:49.720 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T22:36:49.721 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T22:36:49.724 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T22:36:49.725 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T22:36:49.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T22:36:49.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T22:36:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T22:36:49.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T22:36:49.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T22:36:49.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T22:36:49.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T22:36:49.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T22:36:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T22:36:49.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T22:36:49.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T22:36:49.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T22:36:49.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T22:36:49.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T22:36:49.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T22:36:49.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T22:36:49.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T22:36:49.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T22:36:49.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T22:36:49.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T22:36:49.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T22:36:49.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T22:36:49.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T22:36:49.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T22:36:49.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T22:36:49.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:49.751 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.751 DEBUG:teuthology.orchestra.run.smithi145:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T22:36:49.774 INFO:teuthology.orchestra.run.smithi145.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T22:36:49.774 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:for file in vmlinuz System.map config; do 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: fi 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:done 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T22:36:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-09-10T22:36:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T22:36:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T22:36:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-09-10T22:36:49.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.06 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 82.el9 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:36:49.849 INFO:teuthology.orchestra.run.smithi145.stdout:Group : Unspecified 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 8274670 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : builder@centos.org 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : CentOS 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:36:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:36:49.851 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:36:49.851 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2024-09-10T22:36:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:36:49.851 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:36:49.852 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:36:49.852 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:49.852 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:36:50.524 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-10T22:36:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:36:51.916 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-10T22:36:51.918 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-10T22:36:51.937 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.Tf0LqQLJn8 2024-09-10T22:36:51.937 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Tf0LqQLJn8 2024-09-10T22:36:52.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.Tf0LqQLJn8 2024-09-10T22:36:52.152 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.Tf0LqQLJn8 is 6KB 2024-09-10T22:36:52.201 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.Tf0LqQLJn8 2024-09-10T22:36:52.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:36:52.278 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:36:52.278 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:36:52.278 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:36:52.278 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T22:36:52.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:36:52.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-09-10T22:37:03.849 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T22:37:03.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-10T22:37:03.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:07.637 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T22:37:07.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:37:07.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:22.449 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-10T22:37:22.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:37:22.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:26.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-10T22:37:35.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:37:35.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:38.305 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-10T22:37:40.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-10T22:37:49.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:37:49.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:50.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:37:50.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:37:52.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-10T22:38:03.869 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T22:38:04.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:38:04.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:38:08.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-10T22:38:12.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-10T22:38:12.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:38:20.345 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-10T22:38:20.843 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-10T22:38:20.843 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T22:38:20.843 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-10T22:38:20.877 INFO:teuthology.orchestra.run.smithi092.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:20.877 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:20.877 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:38:20.877 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:38:21.877 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T22:38:21.878 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T22:38:21.986 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-10T22:38:22.022 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:23.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:38:23.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:38:26.529 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-10T22:38:27.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:38:27.033 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T22:38:27.033 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-10T22:38:27.068 INFO:teuthology.orchestra.run.smithi067.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:27.068 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:27.068 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:38:27.068 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:38:28.069 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T22:38:28.070 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T22:38:28.163 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-09-10T22:38:28.195 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:36.777 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-10T22:38:37.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:38:37.344 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T22:38:37.344 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-10T22:38:37.378 INFO:teuthology.orchestra.run.smithi145.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:37.379 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T22:38:37.379 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:38:37.379 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:38:38.379 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-10T22:38:38.380 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T22:38:38.484 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-09-10T22:38:38.515 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:38.515 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T22:38:38.527 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T22:38:38.528 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:38:38.533 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:38:38.538 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:38:38.573 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T22:38:38.649 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T22:38:38.800 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T22:38:38.800 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:38:38.804 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:38:38.808 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:38:38.858 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T22:38:38.942 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T22:38:38.942 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:38:38.946 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:38:38.950 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:38:38.999 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.013 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.013 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.036 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:38:39.040 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T22:38:39.305 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T22:38:39.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:38:39.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:38:39.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:38:39.369 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T22:38:39.430 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T22:38:39.430 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:38:39.434 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:38:39.438 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:38:39.473 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:39.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:39.685 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:39.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:39.820 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:38:39.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:38:39.876 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:39.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:40.040 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:40.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:40.172 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:38:40.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:38:40.228 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:40.295 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:38:40.409 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:40.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:38:40.543 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:38:40.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:38:40.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-09-10T22:38:40.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-10T22:38:40.620 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-10T22:38:40.693 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:38:40.764 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:38:41.182 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T22:38:41.190 INFO:teuthology.task.internal:Starting timer... 2024-09-10T22:38:41.190 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T22:38:41.429 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T22:38:41.571 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T22:38:41.571 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-09-10T22:38:41.732 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-09-10T22:38:41.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-10T22:38:41.840 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-10T22:38:41.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-09-10T22:38:41.943 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-09-10T22:38:41.946 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T22:38:41.946 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-09-10T22:38:42.141 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-09-10T22:38:42.142 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-10T22:38:42.189 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-10T22:38:42.189 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-09-10T22:38:42.281 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-09-10T22:38:42.281 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-09-10T22:38:42.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:38:42.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:42.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:38:42.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:42.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:38:42.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:42.458 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T22:38:42.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:38:42.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:38:42.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:38:42.677 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T22:38:42.686 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T22:38:42.699 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T22:38:42.700 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T22:43:25.221 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-09-10T22:43:25.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:43:25.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:43:25.318 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-10T22:43:25.434 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-10T22:43:25.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-10T22:43:25.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:43:25.526 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-10T22:43:25.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-10T22:43:25.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:43:25.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:43:25.737 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-10T22:43:25.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T22:43:25.853 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T22:43:25.864 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T22:43:25.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:43:25.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:43:25.869 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:43:25.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:43:25.872 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:43:25.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:43:25.928 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:43:25.929 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:43:25.956 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:43:25.958 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:43:25.958 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:43:25.986 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:43:26.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-10T22:43:26.010 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-09-10T22:43:26.032 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-10T22:43:26.035 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-09-10T22:43:26.036 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-09-10T22:43:26.060 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-09-10T22:43:26.061 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:43:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:43:26.089 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:43:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:43:26.094 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:43:26.119 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:43:26.183 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.190 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.201 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.216 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.223 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:43:26.224 INFO:teuthology.run_tasks:Running task install... 2024-09-10T22:43:26.232 DEBUG:teuthology.task.install:project ceph 2024-09-10T22:43:26.232 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T22:43:26.232 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T22:43:26.232 INFO:teuthology.task.install:Using flavor: default 2024-09-10T22:43:26.237 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T22:43:26.237 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T22:43:26.237 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:43:26.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:43:26.240 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:43:26.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:43:26.244 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:43:26.245 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:43:26.394 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:43:26.394 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:43:26.410 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:43:26.410 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:43:26.411 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:43:26.411 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:43:26.521 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:43:26.521 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:43:26.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:43:26.533 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:43:26.533 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:43:26.533 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:43:26.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:43:26.537 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:43:26.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:43:26.578 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:43:26.578 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:43:26.589 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:43:26.590 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:43:26.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:43:26.595 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:43:26.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:43:26.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:43:26.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:43:26.755 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:43:26.765 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:43:26.770 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:43:26.810 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-09-10T22:43:26.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-09-10T22:43:26.820 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-09-10T22:43:26.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-09-10T22:43:26.825 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-10T22:43:26.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-10T22:43:27.089 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-09-10T22:43:27.095 INFO:teuthology.orchestra.run.smithi145.stdout:45 files removed 2024-09-10T22:43:27.111 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-10T22:43:27.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:43:27.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:43:27.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:43:27.813 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-10T22:43:27.815 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-09-10T22:43:27.832 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 281 kB/s | 84 kB 00:00 2024-09-10T22:43:28.105 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-10T22:43:28.116 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T22:43:28.133 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T22:43:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T22:43:28.417 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-10T22:43:28.417 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-10T22:43:29.108 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-10T22:43:29.914 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.2 MB 00:01 2024-09-10T22:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-10T22:43:31.948 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-10T22:43:35.708 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.2 MB 00:07 2024-09-10T22:43:36.187 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.5 MB 00:01 2024-09-10T22:43:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-10T22:43:37.794 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T22:43:38.546 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T22:43:39.367 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-10T22:43:39.584 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-10T22:43:40.504 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T22:43:43.003 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.5 MB 00:01 2024-09-10T22:43:44.795 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-10T22:43:45.791 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T22:43:46.130 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-10T22:43:47.185 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-10T22:43:48.753 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:48.767 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:43:48.769 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:43:48.770 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:43:48.771 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:43:48.772 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:43:48.775 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:43:48.776 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:43:48.842 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:43:48.866 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:43:48.867 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:43:48.874 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:43:48.875 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:43:48.876 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:43:48.877 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:43:48.878 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:Install 149 Packages 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 240 M 2024-09-10T22:43:48.879 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-10T22:43:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:49.843 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:43:49.859 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:43:49.860 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:43:49.861 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:43:49.863 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:43:49.866 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:43:49.867 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:43:49.890 INFO:teuthology.orchestra.run.smithi092.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 43 kB/s | 6.6 kB 00:00 2024-09-10T22:43:49.939 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-10T22:43:49.961 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-10T22:43:49.961 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-10T22:43:49.961 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-10T22:43:49.961 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-10T22:43:49.962 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:43:49.962 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:43:49.962 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:49.962 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:43:49.962 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:43:49.963 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:43:49.964 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:49.965 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:43:49.966 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:43:49.967 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:43:49.968 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:43:49.969 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:43:49.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:43:49.971 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:43:49.972 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:43:49.973 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:43:49.974 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:43:49.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:43:49.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:43:49.977 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:43:49.978 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:43:49.979 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:43:49.979 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:43:49.979 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:43:49.979 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout:Install 149 Packages 2024-09-10T22:43:49.980 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 8 Packages 2024-09-10T22:43:49.981 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:43:49.981 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 240 M 2024-09-10T22:43:49.982 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-10T22:43:50.107 INFO:teuthology.orchestra.run.smithi092.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.1 MB/s | 900 kB 00:00 2024-09-10T22:43:50.216 INFO:teuthology.orchestra.run.smithi092.stdout:(3/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-10T22:43:50.241 INFO:teuthology.orchestra.run.smithi092.stdout:(4/157): ceph-immutable-object-cache-19.3.0-485 1.1 MB/s | 145 kB 00:00 2024-09-10T22:43:50.357 INFO:teuthology.orchestra.run.smithi092.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 2.4 MB 00:00 2024-09-10T22:43:50.399 INFO:teuthology.orchestra.run.smithi092.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-09-10T22:43:50.612 INFO:teuthology.orchestra.run.smithi067.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-09-10T22:43:50.641 INFO:teuthology.orchestra.run.smithi092.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 4.8 MB 00:00 2024-09-10T22:43:51.179 INFO:teuthology.orchestra.run.smithi067.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.6 MB/s | 900 kB 00:00 2024-09-10T22:43:51.245 INFO:teuthology.orchestra.run.smithi067.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 2.1 MB/s | 145 kB 00:00 2024-09-10T22:43:51.250 INFO:teuthology.orchestra.run.smithi092.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 14 MB/s | 22 MB 00:01 2024-09-10T22:43:51.309 INFO:teuthology.orchestra.run.smithi092.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 433 kB/s | 25 kB 00:00 2024-09-10T22:43:51.442 INFO:teuthology.orchestra.run.smithi092.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 14 MB/s | 11 MB 00:00 2024-09-10T22:43:51.621 INFO:teuthology.orchestra.run.smithi067.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 4.4 MB/s | 5.5 MB 00:01 2024-09-10T22:43:51.634 INFO:teuthology.orchestra.run.smithi092.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 14 MB/s | 17 MB 00:01 2024-09-10T22:43:51.684 INFO:teuthology.orchestra.run.smithi092.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 931 kB/s | 225 kB 00:00 2024-09-10T22:43:51.727 INFO:teuthology.orchestra.run.smithi092.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 295 kB/s | 27 kB 00:00 2024-09-10T22:43:51.754 INFO:teuthology.orchestra.run.smithi067.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 1.5 MB 00:00 2024-09-10T22:43:51.768 INFO:teuthology.orchestra.run.smithi092.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 380 kB/s | 32 kB 00:00 2024-09-10T22:43:51.810 INFO:teuthology.orchestra.run.smithi092.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.9 MB/s | 755 kB 00:00 2024-09-10T22:43:51.835 INFO:teuthology.orchestra.run.smithi092.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.4 MB/s | 162 kB 00:00 2024-09-10T22:43:51.869 INFO:teuthology.orchestra.run.smithi092.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.1 MB/s | 127 kB 00:00 2024-09-10T22:43:51.936 INFO:teuthology.orchestra.run.smithi092.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 4.9 MB/s | 507 kB 00:00 2024-09-10T22:43:52.013 INFO:teuthology.orchestra.run.smithi067.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-09-10T22:43:52.045 INFO:teuthology.orchestra.run.smithi092.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 418 kB/s | 45 kB 00:00 2024-09-10T22:43:52.128 INFO:teuthology.orchestra.run.smithi092.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.7 MB/s | 143 kB 00:00 2024-09-10T22:43:52.130 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-10T22:43:52.212 INFO:teuthology.orchestra.run.smithi092.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.9 MB/s | 161 kB 00:00 2024-09-10T22:43:52.247 INFO:teuthology.orchestra.run.smithi067.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 9.7 MB/s | 4.8 MB 00:00 2024-09-10T22:43:52.312 INFO:teuthology.orchestra.run.smithi092.stdout:(22/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-10T22:43:52.346 INFO:teuthology.orchestra.run.smithi092.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 2.3 MB/s | 321 kB 00:00 2024-09-10T22:43:52.397 INFO:teuthology.orchestra.run.smithi092.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.6 MB/s | 306 kB 00:00 2024-09-10T22:43:52.430 INFO:teuthology.orchestra.run.smithi092.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.2 MB/s | 99 kB 00:00 2024-09-10T22:43:52.472 INFO:teuthology.orchestra.run.smithi092.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-10T22:43:52.530 INFO:teuthology.orchestra.run.smithi092.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T22:43:52.639 INFO:teuthology.orchestra.run.smithi092.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 316 kB/s | 34 kB 00:00 2024-09-10T22:43:52.672 INFO:teuthology.orchestra.run.smithi067.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 9.4 MB/s | 22 MB 00:02 2024-09-10T22:43:52.706 INFO:teuthology.orchestra.run.smithi092.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.2 MB 00:00 2024-09-10T22:43:52.714 INFO:teuthology.orchestra.run.smithi067.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 608 kB/s | 25 kB 00:00 2024-09-10T22:43:52.731 INFO:teuthology.orchestra.run.smithi092.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.6 MB/s | 154 kB 00:00 2024-09-10T22:43:53.106 INFO:teuthology.orchestra.run.smithi067.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 13 MB/s | 11 MB 00:00 2024-09-10T22:43:53.214 INFO:teuthology.orchestra.run.smithi067.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.0 MB/s | 225 kB 00:00 2024-09-10T22:43:53.289 INFO:teuthology.orchestra.run.smithi067.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 362 kB/s | 27 kB 00:00 2024-09-10T22:43:53.365 INFO:teuthology.orchestra.run.smithi067.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 423 kB/s | 32 kB 00:00 2024-09-10T22:43:53.406 INFO:teuthology.orchestra.run.smithi092.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 11 MB/s | 7.4 MB 00:00 2024-09-10T22:43:53.432 INFO:teuthology.orchestra.run.smithi067.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 11 MB/s | 755 kB 00:00 2024-09-10T22:43:53.481 INFO:teuthology.orchestra.run.smithi092.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.8 MB/s | 291 kB 00:00 2024-09-10T22:43:53.507 INFO:teuthology.orchestra.run.smithi067.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T22:43:53.591 INFO:teuthology.orchestra.run.smithi067.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.5 MB/s | 127 kB 00:00 2024-09-10T22:43:53.658 INFO:teuthology.orchestra.run.smithi067.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 7.4 MB/s | 507 kB 00:00 2024-09-10T22:43:53.665 INFO:teuthology.orchestra.run.smithi092.stdout:(33/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 15 MB/s | 14 MB 00:00 2024-09-10T22:43:53.732 INFO:teuthology.orchestra.run.smithi092.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 197 kB/s | 49 kB 00:00 2024-09-10T22:43:53.782 INFO:teuthology.orchestra.run.smithi092.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 141 kB/s | 16 kB 00:00 2024-09-10T22:43:53.816 INFO:teuthology.orchestra.run.smithi092.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.3 MB/s | 285 kB 00:00 2024-09-10T22:43:53.866 INFO:teuthology.orchestra.run.smithi092.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 9.1 MB/s | 783 kB 00:00 2024-09-10T22:43:53.908 INFO:teuthology.orchestra.run.smithi092.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 199 kB/s | 18 kB 00:00 2024-09-10T22:43:54.008 INFO:teuthology.orchestra.run.smithi067.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 16 MB/s | 5.5 MB 00:00 2024-09-10T22:43:54.017 INFO:teuthology.orchestra.run.smithi092.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-10T22:43:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 907 kB/s | 45 kB 00:00 2024-09-10T22:43:54.092 INFO:teuthology.orchestra.run.smithi092.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-10T22:43:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-10T22:43:54.118 INFO:teuthology.orchestra.run.smithi092.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-10T22:43:54.168 INFO:teuthology.orchestra.run.smithi092.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-10T22:43:54.184 INFO:teuthology.orchestra.run.smithi067.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.4 MB/s | 161 kB 00:00 2024-09-10T22:43:54.218 INFO:teuthology.orchestra.run.smithi092.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-10T22:43:54.242 INFO:teuthology.orchestra.run.smithi067.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 5.4 MB/s | 321 kB 00:00 2024-09-10T22:43:54.243 INFO:teuthology.orchestra.run.smithi092.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-10T22:43:54.285 INFO:teuthology.orchestra.run.smithi092.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T22:43:54.311 INFO:teuthology.orchestra.run.smithi092.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T22:43:54.329 INFO:teuthology.orchestra.run.smithi067.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.5 MB/s | 306 kB 00:00 2024-09-10T22:43:54.353 INFO:teuthology.orchestra.run.smithi092.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-10T22:43:54.376 INFO:teuthology.orchestra.run.smithi067.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 2.1 MB/s | 99 kB 00:00 2024-09-10T22:43:54.427 INFO:teuthology.orchestra.run.smithi067.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T22:43:54.595 INFO:teuthology.orchestra.run.smithi092.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T22:43:54.628 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:54.629 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:43:54.643 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:43:54.644 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:43:54.645 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:43:54.646 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:43:54.648 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:43:54.651 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:43:54.652 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:43:54.652 INFO:teuthology.orchestra.run.smithi067.stdout:(26/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 14 MB/s | 3.2 MB 00:00 2024-09-10T22:43:54.718 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-10T22:43:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.2 MB/s | 171 kB 00:00 2024-09-10T22:43:54.737 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:43:54.738 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:43:54.739 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:43:54.740 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:43:54.741 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:43:54.742 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:43:54.743 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:43:54.744 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:43:54.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:43:54.746 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:43:54.747 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:43:54.748 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:43:54.749 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:43:54.750 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout:Install 149 Packages 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 8 Packages 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:43:54.751 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 240 M 2024-09-10T22:43:54.752 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-10T22:43:54.769 INFO:teuthology.orchestra.run.smithi067.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 825 kB/s | 34 kB 00:00 2024-09-10T22:43:54.819 INFO:teuthology.orchestra.run.smithi067.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T22:43:55.062 INFO:teuthology.orchestra.run.smithi092.stdout:(49/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 13 MB/s | 50 MB 00:03 2024-09-10T22:43:55.087 INFO:teuthology.orchestra.run.smithi092.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 217 kB/s | 107 kB 00:00 2024-09-10T22:43:55.138 INFO:teuthology.orchestra.run.smithi092.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-10T22:43:55.237 INFO:teuthology.orchestra.run.smithi092.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 332 kB/s | 33 kB 00:00 2024-09-10T22:43:55.279 INFO:teuthology.orchestra.run.smithi092.stdout:(53/157): elfutils-libelf-devel-0.191-4.el9.x86 100 kB/s | 22 kB 00:00 2024-09-10T22:43:55.305 INFO:teuthology.orchestra.run.smithi092.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-09-10T22:43:55.347 INFO:teuthology.orchestra.run.smithi092.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-09-10T22:43:55.408 INFO:teuthology.orchestra.run.smithi092.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-10T22:43:55.414 INFO:teuthology.orchestra.run.smithi092.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-10T22:43:55.415 INFO:teuthology.orchestra.run.smithi145.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 46 kB/s | 6.6 kB 00:00 2024-09-10T22:43:55.489 INFO:teuthology.orchestra.run.smithi092.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-09-10T22:43:55.531 INFO:teuthology.orchestra.run.smithi092.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-10T22:43:55.599 INFO:teuthology.orchestra.run.smithi145.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.8 MB/s | 900 kB 00:00 2024-09-10T22:43:55.656 INFO:teuthology.orchestra.run.smithi092.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-10T22:43:55.682 INFO:teuthology.orchestra.run.smithi145.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.7 MB/s | 145 kB 00:00 2024-09-10T22:43:55.715 INFO:teuthology.orchestra.run.smithi092.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-10T22:43:55.757 INFO:teuthology.orchestra.run.smithi092.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-10T22:43:55.770 INFO:teuthology.orchestra.run.smithi067.stdout:(30/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 16 MB/s | 50 MB 00:03 2024-09-10T22:43:55.799 INFO:teuthology.orchestra.run.smithi092.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-09-10T22:43:55.807 INFO:teuthology.orchestra.run.smithi145.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 10 MB/s | 5.5 MB 00:00 2024-09-10T22:43:55.858 INFO:teuthology.orchestra.run.smithi092.stdout:(64/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-09-10T22:43:55.891 INFO:teuthology.orchestra.run.smithi092.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T22:43:55.937 INFO:teuthology.orchestra.run.smithi067.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 13 MB/s | 14 MB 00:01 2024-09-10T22:43:55.942 INFO:teuthology.orchestra.run.smithi092.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-10T22:43:55.949 INFO:teuthology.orchestra.run.smithi145.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-09-10T22:43:55.967 INFO:teuthology.orchestra.run.smithi092.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-10T22:43:55.991 INFO:teuthology.orchestra.run.smithi145.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-10T22:43:56.054 INFO:teuthology.orchestra.run.smithi067.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.4 MB/s | 291 kB 00:00 2024-09-10T22:43:56.251 INFO:teuthology.orchestra.run.smithi092.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-09-10T22:43:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-09-10T22:43:56.404 INFO:teuthology.orchestra.run.smithi067.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 141 kB/s | 49 kB 00:00 2024-09-10T22:43:56.434 INFO:teuthology.orchestra.run.smithi092.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 4.9 MB/s | 910 kB 00:00 2024-09-10T22:43:56.459 INFO:teuthology.orchestra.run.smithi092.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T22:43:56.462 INFO:teuthology.orchestra.run.smithi067.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 283 kB/s | 16 kB 00:00 2024-09-10T22:43:56.513 INFO:teuthology.orchestra.run.smithi067.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-10T22:43:56.568 INFO:teuthology.orchestra.run.smithi092.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 7.3 MB/s | 4.9 MB 00:00 2024-09-10T22:43:56.583 INFO:teuthology.orchestra.run.smithi145.stdout:(7/157): ceph-common-19.3.0-4854.g8f219c47.el9. 17 MB/s | 22 MB 00:01 2024-09-10T22:43:56.602 INFO:teuthology.orchestra.run.smithi092.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-09-10T22:43:56.627 INFO:teuthology.orchestra.run.smithi092.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T22:43:56.672 INFO:teuthology.orchestra.run.smithi067.stdout:(36/157): ceph-mgr-diskprediction-local-19.3.0- 8.2 MB/s | 7.4 MB 00:00 2024-09-10T22:43:56.685 INFO:teuthology.orchestra.run.smithi145.stdout:(8/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 6.5 MB/s | 4.8 MB 00:00 2024-09-10T22:43:56.702 INFO:teuthology.orchestra.run.smithi092.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 818 kB/s | 61 kB 00:00 2024-09-10T22:43:56.713 INFO:teuthology.orchestra.run.smithi067.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 3.8 MB/s | 783 kB 00:00 2024-09-10T22:43:56.730 INFO:teuthology.orchestra.run.smithi067.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 313 kB/s | 18 kB 00:00 2024-09-10T22:43:56.750 INFO:teuthology.orchestra.run.smithi145.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 387 kB/s | 25 kB 00:00 2024-09-10T22:43:56.769 INFO:teuthology.orchestra.run.smithi092.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-10T22:43:56.867 INFO:teuthology.orchestra.run.smithi092.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.5 MB/s | 248 kB 00:00 2024-09-10T22:43:56.928 INFO:teuthology.orchestra.run.smithi092.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 424 kB/s | 26 kB 00:00 2024-09-10T22:43:56.987 INFO:teuthology.orchestra.run.smithi092.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-09-10T22:43:57.062 INFO:teuthology.orchestra.run.smithi092.stdout:(80/157): postgresql-server-13.16-1.el9.x86_64. 9.2 MB/s | 5.8 MB 00:00 2024-09-10T22:43:57.087 INFO:teuthology.orchestra.run.smithi092.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T22:43:57.154 INFO:teuthology.orchestra.run.smithi145.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 15 MB/s | 17 MB 00:01 2024-09-10T22:43:57.171 INFO:teuthology.orchestra.run.smithi092.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-10T22:43:57.189 INFO:teuthology.orchestra.run.smithi067.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-10T22:43:57.196 INFO:teuthology.orchestra.run.smithi092.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 260 kB/s | 35 kB 00:00 2024-09-10T22:43:57.229 INFO:teuthology.orchestra.run.smithi092.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-10T22:43:57.255 INFO:teuthology.orchestra.run.smithi092.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T22:43:57.289 INFO:teuthology.orchestra.run.smithi092.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T22:43:57.301 INFO:teuthology.orchestra.run.smithi145.stdout:(11/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 15 MB/s | 11 MB 00:00 2024-09-10T22:43:57.314 INFO:teuthology.orchestra.run.smithi092.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-09-10T22:43:57.326 INFO:teuthology.orchestra.run.smithi145.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 1.3 MB/s | 225 kB 00:00 2024-09-10T22:43:57.347 INFO:teuthology.orchestra.run.smithi092.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T22:43:57.360 INFO:teuthology.orchestra.run.smithi145.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 464 kB/s | 27 kB 00:00 2024-09-10T22:43:57.373 INFO:teuthology.orchestra.run.smithi092.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T22:43:57.385 INFO:teuthology.orchestra.run.smithi145.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 544 kB/s | 32 kB 00:00 2024-09-10T22:43:57.406 INFO:teuthology.orchestra.run.smithi092.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-10T22:43:57.435 INFO:teuthology.orchestra.run.smithi145.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 9.8 MB/s | 755 kB 00:00 2024-09-10T22:43:57.457 INFO:teuthology.orchestra.run.smithi092.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-09-10T22:43:57.460 INFO:teuthology.orchestra.run.smithi145.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T22:43:57.481 INFO:teuthology.orchestra.run.smithi067.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 655 kB/s | 191 kB 00:00 2024-09-10T22:43:57.486 INFO:teuthology.orchestra.run.smithi145.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T22:43:57.491 INFO:teuthology.orchestra.run.smithi092.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 634 kB/s | 54 kB 00:00 2024-09-10T22:43:57.519 INFO:teuthology.orchestra.run.smithi145.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 8.5 MB/s | 507 kB 00:00 2024-09-10T22:43:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 991 kB/s | 801 kB 00:00 2024-09-10T22:43:57.565 INFO:teuthology.orchestra.run.smithi092.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-10T22:43:57.578 INFO:teuthology.orchestra.run.smithi145.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 775 kB/s | 45 kB 00:00 2024-09-10T22:43:57.591 INFO:teuthology.orchestra.run.smithi092.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-09-10T22:43:57.616 INFO:teuthology.orchestra.run.smithi092.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-09-10T22:43:57.650 INFO:teuthology.orchestra.run.smithi092.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T22:43:57.653 INFO:teuthology.orchestra.run.smithi145.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-10T22:43:57.675 INFO:teuthology.orchestra.run.smithi092.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-10T22:43:57.709 INFO:teuthology.orchestra.run.smithi092.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 817 kB/s | 47 kB 00:00 2024-09-10T22:43:57.734 INFO:teuthology.orchestra.run.smithi092.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-10T22:43:57.754 INFO:teuthology.orchestra.run.smithi145.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 21 MB/s | 5.5 MB 00:00 2024-09-10T22:43:57.767 INFO:teuthology.orchestra.run.smithi092.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-10T22:43:57.779 INFO:teuthology.orchestra.run.smithi145.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.3 MB/s | 161 kB 00:00 2024-09-10T22:43:57.793 INFO:teuthology.orchestra.run.smithi092.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T22:43:57.813 INFO:teuthology.orchestra.run.smithi145.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 5.4 MB/s | 321 kB 00:00 2024-09-10T22:43:57.826 INFO:teuthology.orchestra.run.smithi092.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T22:43:57.832 INFO:teuthology.orchestra.run.smithi067.stdout:(42/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 3.0 MB/s | 17 MB 00:05 2024-09-10T22:43:57.838 INFO:teuthology.orchestra.run.smithi145.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T22:43:57.852 INFO:teuthology.orchestra.run.smithi092.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-10T22:43:57.857 INFO:teuthology.orchestra.run.smithi067.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 674 kB/s | 253 kB 00:00 2024-09-10T22:43:57.863 INFO:teuthology.orchestra.run.smithi145.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T22:43:57.888 INFO:teuthology.orchestra.run.smithi145.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T22:43:57.899 INFO:teuthology.orchestra.run.smithi067.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-09-10T22:43:57.902 INFO:teuthology.orchestra.run.smithi092.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-10T22:43:57.947 INFO:teuthology.orchestra.run.smithi145.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T22:43:57.952 INFO:teuthology.orchestra.run.smithi092.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-09-10T22:43:57.974 INFO:teuthology.orchestra.run.smithi067.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-10T22:43:58.014 INFO:teuthology.orchestra.run.smithi145.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 21 MB/s | 3.2 MB 00:00 2024-09-10T22:43:58.016 INFO:teuthology.orchestra.run.smithi067.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-10T22:43:58.031 INFO:teuthology.orchestra.run.smithi145.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 411 kB/s | 34 kB 00:00 2024-09-10T22:43:58.036 INFO:teuthology.orchestra.run.smithi092.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-09-10T22:43:58.061 INFO:teuthology.orchestra.run.smithi092.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-10T22:43:58.085 INFO:teuthology.orchestra.run.smithi145.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.1 MB/s | 154 kB 00:00 2024-09-10T22:43:58.094 INFO:teuthology.orchestra.run.smithi092.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.2 MB/s | 309 kB 00:00 2024-09-10T22:43:58.108 INFO:teuthology.orchestra.run.smithi067.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-09-10T22:43:58.120 INFO:teuthology.orchestra.run.smithi092.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T22:43:58.145 INFO:teuthology.orchestra.run.smithi092.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T22:43:58.170 INFO:teuthology.orchestra.run.smithi092.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T22:43:58.192 INFO:teuthology.orchestra.run.smithi067.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-10T22:43:58.212 INFO:teuthology.orchestra.run.smithi092.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-10T22:43:58.233 INFO:teuthology.orchestra.run.smithi067.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-10T22:43:58.275 INFO:teuthology.orchestra.run.smithi067.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-10T22:43:58.301 INFO:teuthology.orchestra.run.smithi067.stdout:(51/157): python3-ply-3.11-14.el9.noarch.rpm 227 kB/s | 106 kB 00:00 2024-09-10T22:43:58.343 INFO:teuthology.orchestra.run.smithi067.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-10T22:43:58.376 INFO:teuthology.orchestra.run.smithi067.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-10T22:43:58.388 INFO:teuthology.orchestra.run.smithi092.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-10T22:43:58.410 INFO:teuthology.orchestra.run.smithi067.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-10T22:43:58.435 INFO:teuthology.orchestra.run.smithi067.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-09-10T22:43:58.438 INFO:teuthology.orchestra.run.smithi092.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-10T22:43:58.463 INFO:teuthology.orchestra.run.smithi092.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-10T22:43:58.485 INFO:teuthology.orchestra.run.smithi067.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-10T22:43:58.488 INFO:teuthology.orchestra.run.smithi092.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T22:43:58.490 INFO:teuthology.orchestra.run.smithi145.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-10T22:43:58.510 INFO:teuthology.orchestra.run.smithi067.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-10T22:43:58.565 INFO:teuthology.orchestra.run.smithi092.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 185 kB/s | 14 kB 00:00 2024-09-10T22:43:58.577 INFO:teuthology.orchestra.run.smithi067.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-10T22:43:58.582 INFO:teuthology.orchestra.run.smithi145.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.1 MB/s | 291 kB 00:00 2024-09-10T22:43:58.595 INFO:teuthology.orchestra.run.smithi067.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-10T22:43:58.620 INFO:teuthology.orchestra.run.smithi067.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T22:43:58.632 INFO:teuthology.orchestra.run.smithi145.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 994 kB/s | 49 kB 00:00 2024-09-10T22:43:58.654 INFO:teuthology.orchestra.run.smithi067.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-10T22:43:58.687 INFO:teuthology.orchestra.run.smithi067.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-10T22:43:58.712 INFO:teuthology.orchestra.run.smithi067.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 267 kB/s | 24 kB 00:00 2024-09-10T22:43:58.737 INFO:teuthology.orchestra.run.smithi067.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-09-10T22:43:58.763 INFO:teuthology.orchestra.run.smithi067.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T22:43:58.764 INFO:teuthology.orchestra.run.smithi092.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 865 kB/s | 172 kB 00:00 2024-09-10T22:43:58.788 INFO:teuthology.orchestra.run.smithi067.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-10T22:43:58.798 INFO:teuthology.orchestra.run.smithi092.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-09-10T22:43:58.813 INFO:teuthology.orchestra.run.smithi067.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T22:43:58.831 INFO:teuthology.orchestra.run.smithi092.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T22:43:58.897 INFO:teuthology.orchestra.run.smithi067.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T22:43:58.906 INFO:teuthology.orchestra.run.smithi092.stdout:(121/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-10T22:43:58.916 INFO:teuthology.orchestra.run.smithi145.stdout:(34/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 16 MB/s | 14 MB 00:00 2024-09-10T22:43:58.932 INFO:teuthology.orchestra.run.smithi092.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-09-10T22:43:58.941 INFO:teuthology.orchestra.run.smithi145.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 53 kB/s | 16 kB 00:00 2024-09-10T22:43:58.957 INFO:teuthology.orchestra.run.smithi092.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-10T22:43:58.966 INFO:teuthology.orchestra.run.smithi145.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-10T22:43:58.972 INFO:teuthology.orchestra.run.smithi067.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-10T22:43:58.982 INFO:teuthology.orchestra.run.smithi092.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-09-10T22:43:59.008 INFO:teuthology.orchestra.run.smithi092.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-10T22:43:59.025 INFO:teuthology.orchestra.run.smithi145.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 9.2 MB/s | 783 kB 00:00 2024-09-10T22:43:59.033 INFO:teuthology.orchestra.run.smithi092.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-10T22:43:59.047 INFO:teuthology.orchestra.run.smithi067.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-10T22:43:59.050 INFO:teuthology.orchestra.run.smithi145.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 219 kB/s | 18 kB 00:00 2024-09-10T22:43:59.058 INFO:teuthology.orchestra.run.smithi092.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-09-10T22:43:59.073 INFO:teuthology.orchestra.run.smithi067.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-10T22:43:59.084 INFO:teuthology.orchestra.run.smithi092.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-10T22:43:59.106 INFO:teuthology.orchestra.run.smithi067.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-10T22:43:59.109 INFO:teuthology.orchestra.run.smithi092.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-10T22:43:59.134 INFO:teuthology.orchestra.run.smithi092.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:43:59.148 INFO:teuthology.orchestra.run.smithi067.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-09-10T22:43:59.174 INFO:teuthology.orchestra.run.smithi067.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T22:43:59.184 INFO:teuthology.orchestra.run.smithi092.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-09-10T22:43:59.201 INFO:teuthology.orchestra.run.smithi145.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-10T22:43:59.210 INFO:teuthology.orchestra.run.smithi092.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-10T22:43:59.232 INFO:teuthology.orchestra.run.smithi067.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T22:43:59.252 INFO:teuthology.orchestra.run.smithi092.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T22:43:59.277 INFO:teuthology.orchestra.run.smithi092.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-10T22:43:59.283 INFO:teuthology.orchestra.run.smithi067.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-10T22:43:59.292 INFO:teuthology.orchestra.run.smithi145.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-10T22:43:59.302 INFO:teuthology.orchestra.run.smithi092.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-10T22:43:59.328 INFO:teuthology.orchestra.run.smithi092.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T22:43:59.351 INFO:teuthology.orchestra.run.smithi145.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-10T22:43:59.353 INFO:teuthology.orchestra.run.smithi092.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-10T22:43:59.366 INFO:teuthology.orchestra.run.smithi067.stdout:(77/157): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-10T22:43:59.376 INFO:teuthology.orchestra.run.smithi145.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-10T22:43:59.391 INFO:teuthology.orchestra.run.smithi067.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T22:43:59.403 INFO:teuthology.orchestra.run.smithi092.stdout:(138/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-09-10T22:43:59.417 INFO:teuthology.orchestra.run.smithi067.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-10T22:43:59.428 INFO:teuthology.orchestra.run.smithi092.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-09-10T22:43:59.435 INFO:teuthology.orchestra.run.smithi145.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T22:43:59.453 INFO:teuthology.orchestra.run.smithi092.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T22:43:59.458 INFO:teuthology.orchestra.run.smithi067.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-09-10T22:43:59.479 INFO:teuthology.orchestra.run.smithi092.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T22:43:59.484 INFO:teuthology.orchestra.run.smithi067.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-10T22:43:59.496 INFO:teuthology.orchestra.run.smithi092.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-10T22:43:59.509 INFO:teuthology.orchestra.run.smithi067.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T22:43:59.510 INFO:teuthology.orchestra.run.smithi145.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-10T22:43:59.529 INFO:teuthology.orchestra.run.smithi092.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-10T22:43:59.555 INFO:teuthology.orchestra.run.smithi092.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-10T22:43:59.578 INFO:teuthology.orchestra.run.smithi145.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T22:43:59.580 INFO:teuthology.orchestra.run.smithi092.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T22:43:59.605 INFO:teuthology.orchestra.run.smithi092.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-09-10T22:43:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-10T22:43:59.619 INFO:teuthology.orchestra.run.smithi145.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-09-10T22:43:59.630 INFO:teuthology.orchestra.run.smithi092.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T22:43:59.635 INFO:teuthology.orchestra.run.smithi067.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-10T22:43:59.656 INFO:teuthology.orchestra.run.smithi092.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T22:43:59.660 INFO:teuthology.orchestra.run.smithi067.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T22:43:59.685 INFO:teuthology.orchestra.run.smithi067.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T22:43:59.710 INFO:teuthology.orchestra.run.smithi067.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-10T22:43:59.714 INFO:teuthology.orchestra.run.smithi092.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T22:43:59.736 INFO:teuthology.orchestra.run.smithi067.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T22:43:59.761 INFO:teuthology.orchestra.run.smithi067.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T22:43:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-09-10T22:43:59.812 INFO:teuthology.orchestra.run.smithi067.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T22:43:59.814 INFO:teuthology.orchestra.run.smithi092.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 23 MB/s | 2.3 MB 00:00 2024-09-10T22:43:59.837 INFO:teuthology.orchestra.run.smithi067.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-09-10T22:43:59.879 INFO:teuthology.orchestra.run.smithi067.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T22:43:59.915 INFO:teuthology.orchestra.run.smithi092.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-09-10T22:43:59.974 INFO:teuthology.orchestra.run.smithi092.stdout:(152/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-10T22:43:59.980 INFO:teuthology.orchestra.run.smithi067.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-10T22:43:59.999 INFO:teuthology.orchestra.run.smithi092.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 490 kB/s | 41 kB 00:00 2024-09-10T22:44:00.004 INFO:teuthology.orchestra.run.smithi067.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 353 kB/s | 44 kB 00:00 2024-09-10T22:44:00.030 INFO:teuthology.orchestra.run.smithi067.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-09-10T22:44:00.047 INFO:teuthology.orchestra.run.smithi067.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 857 kB/s | 36 kB 00:00 2024-09-10T22:44:00.057 INFO:teuthology.orchestra.run.smithi092.stdout:(154/157): librados2-19.3.0-4854.g8f219c47.el9. 8.1 MB/s | 3.5 MB 00:00 2024-09-10T22:44:00.063 INFO:teuthology.orchestra.run.smithi067.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-10T22:44:00.082 INFO:teuthology.orchestra.run.smithi092.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-10T22:44:00.089 INFO:teuthology.orchestra.run.smithi067.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 7.2 MB/s | 304 kB 00:00 2024-09-10T22:44:00.105 INFO:teuthology.orchestra.run.smithi067.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T22:44:00.124 INFO:teuthology.orchestra.run.smithi092.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-10T22:44:00.131 INFO:teuthology.orchestra.run.smithi067.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-10T22:44:00.136 INFO:teuthology.orchestra.run.smithi145.stdout:(47/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 15 MB/s | 50 MB 00:03 2024-09-10T22:44:00.148 INFO:teuthology.orchestra.run.smithi067.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-10T22:44:00.162 INFO:teuthology.orchestra.run.smithi145.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 368 kB/s | 215 kB 00:00 2024-09-10T22:44:00.165 INFO:teuthology.orchestra.run.smithi067.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-10T22:44:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-10T22:44:00.291 INFO:teuthology.orchestra.run.smithi092.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-10T22:44:00.296 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:44:00.296 INFO:teuthology.orchestra.run.smithi092.stdout:Total 21 MB/s | 240 MB 00:11 2024-09-10T22:44:00.312 INFO:teuthology.orchestra.run.smithi145.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 713 kB/s | 107 kB 00:00 2024-09-10T22:44:00.354 INFO:teuthology.orchestra.run.smithi145.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 526 kB/s | 22 kB 00:00 2024-09-10T22:44:00.396 INFO:teuthology.orchestra.run.smithi145.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-10T22:44:00.438 INFO:teuthology.orchestra.run.smithi145.stdout:(52/157): automake-1.16.2-8.el9.noarch.rpm 842 kB/s | 689 kB 00:00 2024-09-10T22:44:00.454 INFO:teuthology.orchestra.run.smithi145.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-09-10T22:44:00.466 INFO:teuthology.orchestra.run.smithi067.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-10T22:44:00.480 INFO:teuthology.orchestra.run.smithi145.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-10T22:44:00.497 INFO:teuthology.orchestra.run.smithi145.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-09-10T22:44:00.507 INFO:teuthology.orchestra.run.smithi067.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-10T22:44:00.522 INFO:teuthology.orchestra.run.smithi145.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-10T22:44:00.541 INFO:teuthology.orchestra.run.smithi067.stdout:(107/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-10T22:44:00.555 INFO:teuthology.orchestra.run.smithi145.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-10T22:44:00.566 INFO:teuthology.orchestra.run.smithi067.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 347 kB/s | 19 kB 00:00 2024-09-10T22:44:00.606 INFO:teuthology.orchestra.run.smithi145.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-10T22:44:00.608 INFO:teuthology.orchestra.run.smithi067.stdout:(109/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-09-10T22:44:00.631 INFO:teuthology.orchestra.run.smithi145.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-10T22:44:00.642 INFO:teuthology.orchestra.run.smithi067.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-09-10T22:44:00.667 INFO:teuthology.orchestra.run.smithi067.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 864 kB/s | 49 kB 00:00 2024-09-10T22:44:00.692 INFO:teuthology.orchestra.run.smithi067.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T22:44:00.698 INFO:teuthology.orchestra.run.smithi145.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-10T22:44:00.732 INFO:teuthology.orchestra.run.smithi145.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 5.7 MB/s | 581 kB 00:00 2024-09-10T22:44:00.742 INFO:teuthology.orchestra.run.smithi067.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-10T22:44:00.749 INFO:teuthology.orchestra.run.smithi145.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-10T22:44:00.766 INFO:teuthology.orchestra.run.smithi145.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-10T22:44:00.800 INFO:teuthology.orchestra.run.smithi145.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T22:44:00.801 INFO:teuthology.orchestra.run.smithi067.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-09-10T22:44:00.826 INFO:teuthology.orchestra.run.smithi067.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T22:44:00.833 INFO:teuthology.orchestra.run.smithi145.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-10T22:44:00.850 INFO:teuthology.orchestra.run.smithi145.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-10T22:44:00.860 INFO:teuthology.orchestra.run.smithi067.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-10T22:44:00.885 INFO:teuthology.orchestra.run.smithi067.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-09-10T22:44:00.911 INFO:teuthology.orchestra.run.smithi067.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-10T22:44:00.944 INFO:teuthology.orchestra.run.smithi067.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T22:44:00.978 INFO:teuthology.orchestra.run.smithi067.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T22:44:00.992 INFO:teuthology.orchestra.run.smithi145.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-10T22:44:01.011 INFO:teuthology.orchestra.run.smithi067.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T22:44:01.037 INFO:teuthology.orchestra.run.smithi067.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T22:44:01.070 INFO:teuthology.orchestra.run.smithi067.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T22:44:01.082 INFO:teuthology.orchestra.run.smithi145.stdout:(68/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-09-10T22:44:01.112 INFO:teuthology.orchestra.run.smithi067.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-09-10T22:44:01.146 INFO:teuthology.orchestra.run.smithi067.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 534 kB/s | 18 kB 00:00 2024-09-10T22:44:01.151 INFO:teuthology.orchestra.run.smithi145.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-10T22:44:01.172 INFO:teuthology.orchestra.run.smithi067.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-10T22:44:01.196 INFO:teuthology.orchestra.run.smithi067.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 791 kB/s | 19 kB 00:00 2024-09-10T22:44:01.201 INFO:teuthology.orchestra.run.smithi145.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T22:44:01.221 INFO:teuthology.orchestra.run.smithi067.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-10T22:44:01.226 INFO:teuthology.orchestra.run.smithi145.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-10T22:44:01.269 INFO:teuthology.orchestra.run.smithi145.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-10T22:44:01.305 INFO:teuthology.orchestra.run.smithi067.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-09-10T22:44:01.310 INFO:teuthology.orchestra.run.smithi145.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-10T22:44:01.335 INFO:teuthology.orchestra.run.smithi145.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T22:44:01.338 INFO:teuthology.orchestra.run.smithi067.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-10T22:44:01.394 INFO:teuthology.orchestra.run.smithi145.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T22:44:01.397 INFO:teuthology.orchestra.run.smithi067.stdout:(131/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-10T22:44:01.422 INFO:teuthology.orchestra.run.smithi067.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 945 kB/s | 79 kB 00:00 2024-09-10T22:44:01.478 INFO:teuthology.orchestra.run.smithi145.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-10T22:44:01.503 INFO:teuthology.orchestra.run.smithi145.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T22:44:01.551 INFO:teuthology.orchestra.run.smithi145.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-09-10T22:44:01.580 INFO:teuthology.orchestra.run.smithi145.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-09-10T22:44:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 313 kB/s | 58 kB 00:00 2024-09-10T22:44:01.604 INFO:teuthology.orchestra.run.smithi145.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 366 kB/s | 19 kB 00:00 2024-09-10T22:44:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 514 kB/s | 16 kB 00:00 2024-09-10T22:44:01.639 INFO:teuthology.orchestra.run.smithi067.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T22:44:01.665 INFO:teuthology.orchestra.run.smithi067.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T22:44:01.687 INFO:teuthology.orchestra.run.smithi145.stdout:(81/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T22:44:01.690 INFO:teuthology.orchestra.run.smithi067.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi145.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 318 kB/s | 35 kB 00:00 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi067.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-10T22:44:01.738 INFO:teuthology.orchestra.run.smithi145.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-10T22:44:01.740 INFO:teuthology.orchestra.run.smithi067.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T22:44:01.763 INFO:teuthology.orchestra.run.smithi145.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-09-10T22:44:01.774 INFO:teuthology.orchestra.run.smithi067.stdout:(140/157): python3-kubernetes-26.1.0-2.el9.noar 1.8 MB/s | 1.0 MB 00:00 2024-09-10T22:44:01.800 INFO:teuthology.orchestra.run.smithi067.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-10T22:44:01.814 INFO:teuthology.orchestra.run.smithi145.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-10T22:44:01.825 INFO:teuthology.orchestra.run.smithi067.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-10T22:44:01.839 INFO:teuthology.orchestra.run.smithi145.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-10T22:44:01.850 INFO:teuthology.orchestra.run.smithi067.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-10T22:44:01.856 INFO:teuthology.orchestra.run.smithi145.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T22:44:01.875 INFO:teuthology.orchestra.run.smithi067.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T22:44:01.901 INFO:teuthology.orchestra.run.smithi067.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T22:44:01.918 INFO:teuthology.orchestra.run.smithi067.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-09-10T22:44:01.931 INFO:teuthology.orchestra.run.smithi145.stdout:(88/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-09-10T22:44:01.957 INFO:teuthology.orchestra.run.smithi145.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-10T22:44:01.982 INFO:teuthology.orchestra.run.smithi145.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-10T22:44:02.024 INFO:teuthology.orchestra.run.smithi145.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 766 kB/s | 51 kB 00:00 2024-09-10T22:44:02.026 INFO:teuthology.orchestra.run.smithi067.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-10T22:44:02.041 INFO:teuthology.orchestra.run.smithi145.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-09-10T22:44:02.052 INFO:teuthology.orchestra.run.smithi067.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-09-10T22:44:02.066 INFO:teuthology.orchestra.run.smithi145.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-10T22:44:02.083 INFO:teuthology.orchestra.run.smithi145.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-09-10T22:44:02.100 INFO:teuthology.orchestra.run.smithi145.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-10T22:44:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-10T22:44:02.133 INFO:teuthology.orchestra.run.smithi145.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T22:44:02.150 INFO:teuthology.orchestra.run.smithi145.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-10T22:44:02.167 INFO:teuthology.orchestra.run.smithi145.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-10T22:44:02.201 INFO:teuthology.orchestra.run.smithi145.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T22:44:02.218 INFO:teuthology.orchestra.run.smithi145.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T22:44:02.243 INFO:teuthology.orchestra.run.smithi145.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-10T22:44:02.260 INFO:teuthology.orchestra.run.smithi145.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-10T22:44:02.277 INFO:teuthology.orchestra.run.smithi145.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-10T22:44:02.365 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-10T22:44:02.411 INFO:teuthology.orchestra.run.smithi067.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 9.0 MB/s | 3.2 MB 00:00 2024-09-10T22:44:02.546 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-10T22:44:02.546 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-10T22:44:02.603 INFO:teuthology.orchestra.run.smithi067.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 6.0 MB/s | 3.5 MB 00:00 2024-09-10T22:44:02.836 INFO:teuthology.orchestra.run.smithi145.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-10T22:44:02.886 INFO:teuthology.orchestra.run.smithi067.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-10T22:44:03.053 INFO:teuthology.orchestra.run.smithi145.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-09-10T22:44:03.095 INFO:teuthology.orchestra.run.smithi145.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-10T22:44:03.112 INFO:teuthology.orchestra.run.smithi145.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-10T22:44:03.153 INFO:teuthology.orchestra.run.smithi145.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-10T22:44:03.162 INFO:teuthology.orchestra.run.smithi067.stdout:(153/157): NetworkManager-1.49.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-10T22:44:03.179 INFO:teuthology.orchestra.run.smithi145.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T22:44:03.204 INFO:teuthology.orchestra.run.smithi145.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T22:44:03.229 INFO:teuthology.orchestra.run.smithi145.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T22:44:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 663 kB/s | 249 kB 00:00 2024-09-10T22:44:03.263 INFO:teuthology.orchestra.run.smithi145.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-10T22:44:03.329 INFO:teuthology.orchestra.run.smithi067.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 2.0 MB/s | 1.8 MB 00:00 2024-09-10T22:44:03.346 INFO:teuthology.orchestra.run.smithi145.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-09-10T22:44:03.379 INFO:teuthology.orchestra.run.smithi067.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-09-10T22:44:03.405 INFO:teuthology.orchestra.run.smithi145.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-10T22:44:03.422 INFO:teuthology.orchestra.run.smithi145.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-10T22:44:03.439 INFO:teuthology.orchestra.run.smithi145.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T22:44:03.456 INFO:teuthology.orchestra.run.smithi145.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 840 kB/s | 14 kB 00:00 2024-09-10T22:44:03.548 INFO:teuthology.orchestra.run.smithi145.stdout:(118/157): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-10T22:44:03.581 INFO:teuthology.orchestra.run.smithi145.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-09-10T22:44:03.625 INFO:teuthology.orchestra.run.smithi145.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-09-10T22:44:03.649 INFO:teuthology.orchestra.run.smithi145.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-10T22:44:03.674 INFO:teuthology.orchestra.run.smithi145.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T22:44:03.708 INFO:teuthology.orchestra.run.smithi145.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T22:44:03.733 INFO:teuthology.orchestra.run.smithi145.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-10T22:44:03.767 INFO:teuthology.orchestra.run.smithi145.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 525 kB/s | 18 kB 00:00 2024-09-10T22:44:03.792 INFO:teuthology.orchestra.run.smithi145.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-10T22:44:03.817 INFO:teuthology.orchestra.run.smithi145.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-10T22:44:03.860 INFO:teuthology.orchestra.run.smithi067.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 2.4 MB 00:00 2024-09-10T22:44:03.868 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:44:03.868 INFO:teuthology.orchestra.run.smithi067.stdout:Total 17 MB/s | 240 MB 00:13 2024-09-10T22:44:03.893 INFO:teuthology.orchestra.run.smithi145.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-09-10T22:44:03.935 INFO:teuthology.orchestra.run.smithi145.stdout:(129/157): protobuf-compiler-3.14.0-13.el9.x86_ 521 kB/s | 863 kB 00:01 2024-09-10T22:44:04.010 INFO:teuthology.orchestra.run.smithi145.stdout:(130/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-09-10T22:44:04.036 INFO:teuthology.orchestra.run.smithi145.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.3 MB/s | 79 kB 00:00 2024-09-10T22:44:04.061 INFO:teuthology.orchestra.run.smithi145.stdout:(132/157): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-09-10T22:44:04.086 INFO:teuthology.orchestra.run.smithi145.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T22:44:04.111 INFO:teuthology.orchestra.run.smithi145.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-10T22:44:04.137 INFO:teuthology.orchestra.run.smithi145.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T22:44:04.162 INFO:teuthology.orchestra.run.smithi145.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T22:44:04.187 INFO:teuthology.orchestra.run.smithi145.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T22:44:04.212 INFO:teuthology.orchestra.run.smithi145.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T22:44:04.238 INFO:teuthology.orchestra.run.smithi145.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T22:44:04.263 INFO:teuthology.orchestra.run.smithi145.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T22:44:04.288 INFO:teuthology.orchestra.run.smithi145.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T22:44:04.330 INFO:teuthology.orchestra.run.smithi145.stdout:(142/157): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-09-10T22:44:04.356 INFO:teuthology.orchestra.run.smithi145.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 807 kB/s | 74 kB 00:00 2024-09-10T22:44:04.381 INFO:teuthology.orchestra.run.smithi145.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T22:44:04.425 INFO:teuthology.orchestra.run.smithi145.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-09-10T22:44:04.456 INFO:teuthology.orchestra.run.smithi145.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-10T22:44:04.482 INFO:teuthology.orchestra.run.smithi145.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.3 MB/s | 191 kB 00:00 2024-09-10T22:44:04.515 INFO:teuthology.orchestra.run.smithi145.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-10T22:44:04.590 INFO:teuthology.orchestra.run.smithi145.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T22:44:04.774 INFO:teuthology.orchestra.run.smithi145.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.5 MB 00:00 2024-09-10T22:44:04.833 INFO:teuthology.orchestra.run.smithi145.stdout:(151/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-10T22:44:04.975 INFO:teuthology.orchestra.run.smithi145.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 288 kB/s | 41 kB 00:00 2024-09-10T22:44:05.035 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-10T22:44:05.035 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-10T22:44:05.092 INFO:teuthology.orchestra.run.smithi145.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-10T22:44:05.142 INFO:teuthology.orchestra.run.smithi145.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-09-10T22:44:05.209 INFO:teuthology.orchestra.run.smithi145.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-09-10T22:44:05.242 INFO:teuthology.orchestra.run.smithi145.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-10T22:44:05.468 INFO:teuthology.orchestra.run.smithi145.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-09-10T22:44:05.473 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:44:05.473 INFO:teuthology.orchestra.run.smithi145.stdout:Total 22 MB/s | 240 MB 00:10 2024-09-10T22:44:05.975 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-10T22:44:06.173 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-10T22:44:06.173 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-10T22:44:07.533 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-10T22:44:07.715 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-10T22:44:07.715 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-10T22:44:08.712 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-10T22:44:08.713 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-10T22:44:10.239 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-10T22:44:10.240 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-10T22:44:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-10T22:44:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:44:12.121 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:44:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:44:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:44:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:44:12.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:44:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:44:12.838 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:13.033 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:13.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:13.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:44:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:44:13.515 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:44:13.587 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:44:13.619 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:13.772 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:13.804 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:13.911 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:13.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:13.978 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:14.231 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:14.429 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:44:14.716 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:14.838 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:15.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:44:15.318 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:44:15.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:44:15.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:44:15.506 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:44:15.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:44:15.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:44:15.639 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:44:15.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:44:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:44:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:44:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:44:15.939 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:44:16.020 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:44:16.063 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:44:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-10T22:44:16.115 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:44:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:44:16.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:44:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:44:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:44:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:44:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:44:16.543 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:44:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:44:16.623 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:44:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:44:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:44:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:44:17.020 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-10T22:44:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:44:17.164 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:44:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:44:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:44:17.393 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:44:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:17.420 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:44:17.438 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:44:17.464 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:17.546 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:44:17.584 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:17.591 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:44:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:44:17.622 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:44:17.657 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:44:17.679 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:44:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:44:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:44:17.889 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:44:17.966 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:44:17.991 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:18.056 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:44:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:44:18.120 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:44:18.150 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:18.173 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:18.232 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:44:18.258 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:18.284 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:18.297 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:44:18.319 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:44:18.330 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:18.349 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:44:18.430 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:44:18.466 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:44:18.480 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:44:18.513 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:44:18.525 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:44:18.561 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:44:18.570 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:44:18.660 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:44:18.784 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:18.799 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:44:18.830 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:44:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:44:18.886 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:44:18.912 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:44:18.941 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:44:19.031 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:44:19.056 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:44:19.063 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:19.071 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:44:19.137 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:44:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:44:19.164 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:19.176 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:44:19.203 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:44:19.205 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:44:19.322 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:44:19.464 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:19.475 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:44:19.560 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:44:19.597 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:44:19.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:44:19.640 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:44:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:44:19.680 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:44:19.685 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:44:19.704 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:44:19.719 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:44:19.757 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:44:19.791 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:44:19.799 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:44:19.825 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:44:19.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:44:19.867 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:44:19.896 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:44:19.900 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:44:19.910 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:44:19.965 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:44:19.999 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:44:20.003 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:44:20.032 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:44:20.116 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:44:20.129 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:44:20.131 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:20.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:44:20.190 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:44:20.232 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:44:20.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:44:20.279 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:44:20.333 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:44:20.372 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:44:20.411 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:44:20.444 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:44:20.455 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:44:20.508 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:44:20.593 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:44:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:44:20.637 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:44:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:44:20.681 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:44:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:44:20.763 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:44:20.835 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:44:20.881 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:44:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:44:20.935 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:44:20.974 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:44:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:44:21.024 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:44:21.136 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:44:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:44:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:44:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:44:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:44:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:44:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:44:21.397 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:44:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:44:21.432 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:44:21.476 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:44:21.493 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:44:21.504 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:44:21.547 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:44:21.592 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:44:21.650 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:44:21.656 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:44:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:44:21.696 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:44:21.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:44:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:44:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:44:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:44:21.914 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:44:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:44:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:44:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:44:21.975 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:44:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:44:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:44:22.213 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:44:22.277 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:44:22.403 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:44:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:44:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:44:22.604 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:44:22.611 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:44:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:44:22.700 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:44:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:44:22.738 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:44:22.750 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:44:22.782 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:44:22.820 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:44:22.820 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:44:22.873 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:44:22.881 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:44:22.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:44:23.056 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:44:23.059 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:44:23.104 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:44:23.123 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:44:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:44:23.242 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:44:23.308 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:44:23.311 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:44:23.354 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:44:23.392 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:44:23.405 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:44:23.433 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:44:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:44:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:44:23.469 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:44:23.529 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:44:23.571 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:44:23.608 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:44:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:44:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:44:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:44:23.817 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:44:23.853 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:44:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:44:23.925 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:44:23.930 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:44:23.976 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:44:24.011 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:44:24.060 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:44:24.092 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:44:24.096 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:44:24.134 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:44:24.139 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:44:24.181 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:44:24.184 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:44:24.214 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:44:24.225 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:44:24.239 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:44:24.257 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:44:24.358 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:44:24.395 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:44:24.449 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:44:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:44:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:44:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:44:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:44:24.614 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:44:24.625 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:44:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:44:24.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:44:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:44:24.716 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:44:24.724 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:44:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:44:24.756 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:44:24.793 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:44:24.844 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:44:24.878 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:44:24.921 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:44:24.962 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:44:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:44:25.109 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:44:25.162 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:44:25.165 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:25.208 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:44:25.328 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:44:25.387 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:44:25.583 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:25.617 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:44:25.666 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:44:25.682 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:44:25.733 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:44:25.767 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:44:25.903 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:44:25.975 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:44:26.029 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:44:26.136 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:44:26.193 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:44:26.200 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:44:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:26.260 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:44:26.314 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:44:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:44:27.051 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:44:27.051 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:44:27.053 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:44:27.053 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:44:27.053 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:44:27.053 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:44:27.090 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:44:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:44:27.577 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:44:27.882 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:44:28.018 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:44:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:29.262 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:44:29.361 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:44:29.397 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:44:29.430 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:44:29.669 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:29.676 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:44:29.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:44:29.799 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:44:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:29.888 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:29.888 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:44:29.888 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:44:29.888 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:29.953 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:29.996 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:30.031 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:44:30.080 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:44:30.095 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:44:30.173 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:30.271 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:44:30.306 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:44:30.354 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:44:30.664 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:44:30.711 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:30.785 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:30.812 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:44:30.897 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:44:30.928 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:31.019 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:44:31.078 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:44:33.466 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:33.508 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:33.612 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:33.651 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:34.182 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:34.287 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:44:34.346 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:34.370 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:34.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:44:34.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:44:34.370 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:34.406 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:34.476 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:44:34.476 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:44:34.476 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:44:34.477 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:34.522 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:42.425 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:42.425 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-10T22:44:42.425 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-10T22:44:42.425 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-10T22:44:42.425 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-10T22:44:42.426 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-10T22:44:42.426 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-10T22:44:42.426 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-10T22:44:42.426 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:42.644 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:42.682 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:42.682 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:42.682 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:44:42.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:44.364 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:44.364 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:44:44.365 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:44.365 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:44.666 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:44.666 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:44:44.666 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:44.666 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:45.215 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:45.215 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:44:45.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:45.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:44:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:44:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:44:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:44:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:44:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:44:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:44:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:44:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:44:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:44:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:44:46.727 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:46.749 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:44:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:44:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-09-10T22:44:47.262 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-09-10T22:44:47.263 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-09-10T22:44:47.263 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-09-10T22:44:47.263 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:47.465 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:47.499 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:47.500 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:44:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-09-10T22:44:47.522 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:47.779 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:44:47.779 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:47.779 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:44:47.779 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:49.169 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:49.227 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:49.227 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:44:49.227 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:49.227 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:49.227 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:49.486 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:44:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:49.820 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:44:49.872 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:50.287 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:44:50.346 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:44:50.388 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:44:50.439 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:50.597 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:44:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:44:50.658 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:44:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:44:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:44:50.776 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:44:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:44:50.967 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:44:51.026 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:44:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:44:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:44:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:44:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:44:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:44:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:44:51.400 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:44:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:44:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:44:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:44:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:51.722 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:51.727 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:51.755 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:51.788 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:44:51.869 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:44:51.877 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:51.902 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:52.049 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:44:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:52.130 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:52.130 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:44:52.131 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:52.131 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:52.152 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:52.189 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:44:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:52.232 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:44:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:44:52.316 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:44:52.345 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:52.373 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:44:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:44:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:53.541 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:53.590 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:53.590 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:44:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:53.642 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:53.692 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:53.692 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:53.692 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:44:53.692 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:53.692 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:44:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:44:54.086 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:54.086 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:44:54.086 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:44:54.086 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:44:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:44:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:55.583 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:56.206 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:44:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:56.663 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:56.699 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:56.850 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:57.031 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:44:57.060 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:57.111 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:57.111 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:57.111 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:44:57.111 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:57.111 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:57.112 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:57.169 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:57.195 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:57.303 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:44:57.365 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:44:57.431 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:44:57.459 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:57.619 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:44:57.799 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:44:57.845 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:44:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:57.989 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:44:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:44:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:44:58.389 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:44:58.430 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:58.480 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:58.481 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:58.481 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:44:58.481 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:58.481 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:58.481 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:58.530 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:58.586 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:58.586 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:58.586 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:44:58.587 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:58.639 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:58.690 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:59.031 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:44:59.082 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:44:59.083 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:59.083 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:44:59.083 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:44:59.083 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:44:59.083 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:44:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:44:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:44:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:44:59.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:44:59.266 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:44:59.298 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:44:59.306 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:44:59.359 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:59.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:59.475 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:44:59.475 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:59.475 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:44:59.475 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:44:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:44:59.585 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:44:59.587 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:44:59.617 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:44:59.648 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:44:59.709 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:44:59.950 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:45:00.002 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:45:00.481 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:01.869 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:45:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:45:02.039 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:45:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:45:02.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:45:02.222 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:45:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:45:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:45:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:02.887 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:02.978 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:03.196 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:03.196 INFO:teuthology.orchestra.run.smithi092.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T22:45:03.196 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:45:04.229 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:45:04.270 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:45:04.321 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:45:04.351 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:45:04.386 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:45:04.611 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:45:04.667 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:45:04.697 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:45:04.728 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:45:04.763 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:45:05.407 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:45:05.464 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:45:05.509 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:45:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:45:05.545 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:05.580 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:45:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:05.635 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:05.870 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:45:05.918 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:45:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:45:06.013 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:45:06.073 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:45:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:06.863 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:06.890 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:45:06.906 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:45:07.000 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:45:07.057 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:45:07.162 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:45:07.194 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:45:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:45:07.255 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:45:07.256 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:07.372 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:07.372 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:07.464 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:07.488 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:45:07.503 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:07.504 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:07.749 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:07.749 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:07.883 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:07.956 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:08.176 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:08.176 INFO:teuthology.orchestra.run.smithi145.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T22:45:08.176 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:45:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:45:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:45:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:45:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:45:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:45:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:45:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:45:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:45:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:45:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:45:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:45:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:45:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:45:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:45:09.611 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:45:09.612 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:45:09.613 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:45:09.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:45:09.615 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:45:09.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:45:09.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:45:09.618 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:45:09.619 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:45:09.620 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:45:09.621 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:45:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:45:09.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:45:09.624 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:09.624 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T22:45:09.624 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-10T22:45:12.435 DEBUG:teuthology.parallel:result is None 2024-09-10T22:45:14.848 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:45:14.849 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:45:14.850 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:45:14.851 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:45:14.863 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:45:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:45:14.865 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:45:14.866 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:45:14.867 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:45:14.868 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:45:14.869 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:45:14.870 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:45:14.871 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:45:14.872 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:45:14.873 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:45:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:45:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:45:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:45:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:45:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:45:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:45:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:45:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:45:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:45:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:45:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:45:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:45:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:45:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:45:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:17.061 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.062 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.063 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:45:17.064 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:45:17.065 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:45:17.066 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:45:17.067 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:45:17.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:45:17.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:45:17.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:45:17.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:45:17.072 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:45:17.073 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:45:17.074 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:45:17.074 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:45:17.074 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-10T22:45:17.703 DEBUG:teuthology.parallel:result is None 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:18.470 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.471 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.472 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:45:18.473 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:45:18.474 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:45:18.475 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:45:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:45:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:45:18.478 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:45:18.479 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:45:18.480 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:45:18.481 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:45:18.482 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:45:18.483 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:45:18.483 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:45:18.483 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-10T22:45:19.175 DEBUG:teuthology.parallel:result is None 2024-09-10T22:45:19.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:19.326 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:45:19.371 INFO:teuthology.orchestra.run.smithi067.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.372 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.372 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:45:19.374 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:19.517 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:45:19.567 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.567 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.568 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:45:19.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:19.712 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:45:19.758 INFO:teuthology.orchestra.run.smithi145.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.758 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:45:19.758 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:45:19.760 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T22:45:19.760 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:45:19.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:45:19.824 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:45:19.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:45:19.887 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:45:19.887 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:45:19.945 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T22:45:19.945 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:45:19.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:45:20.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:45:20.105 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:45:20.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:45:20.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:45:20.259 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:45:20.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:45:20.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:45:20.412 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T22:45:20.413 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:45:20.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:45:20.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:45:20.571 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:45:20.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:45:20.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:45:20.720 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:45:20.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:45:20.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:45:20.873 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T22:45:20.874 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:45:20.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:45:20.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:45:21.030 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:45:21.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:45:21.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:45:21.182 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:45:21.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:45:21.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:45:21.338 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T22:45:21.434 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T22:45:21.434 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:21.435 INFO:tasks.cephadm:Cluster fsid is 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:45:21.435 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T22:45:21.435 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.145'} 2024-09-10T22:45:21.435 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-09-10T22:45:21.435 INFO:tasks.cephadm:First mgr is x 2024-09-10T22:45:21.435 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T22:45:21.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-09-10T22:45:21.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-09-10T22:45:21.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-09-10T22:45:21.608 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T22:45:21.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:21.757 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T22:45:21.907 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:45:21.909 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:45:21.909 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:45:21.909 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:22.333 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:45 /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:22.333 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:22.767 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:45 /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:22.767 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:23.204 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:45 /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:23.204 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:23.247 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:23.287 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:45:23.402 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T22:45:23.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:45:23.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:45:23.411 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:45:25.289 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:45:25.471 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:45:25.505 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:45:59.238 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-10T22:45:59.238 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:45:59.238 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:45:59.238 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-09-10T22:45:59.238 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:45:59.239 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-10T22:45:59.239 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-10T22:45:59.420 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-10T22:46:00.086 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T22:46:00.087 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:46:00.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-09-10T22:46:00.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-09-10T22:46:00.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-09-10T22:46:00.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-09-10T22:46:00.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-09-10T22:46:00.424 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-09-10T22:46:00.480 INFO:tasks.cephadm:Writing seed config... 2024-09-10T22:46:00.481 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T22:46:00.482 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T22:46:00.483 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T22:46:00.484 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:46:00.484 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T22:46:00.519 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-10T22:46:00.520 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T22:46:00.563 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T22:46:00.608 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T22:46:00.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:46:00.874 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:46:00.874 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-09-10T22:46:00.875 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T22:46:00.875 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-09-10T22:46:00.919 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:46:00.919 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-09-10T22:46:00.920 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-09-10T22:46:00.933 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:46:00.933 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:46:00.946 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:46:00.946 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:46:00.960 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-09-10T22:46:00.972 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-09-10T22:46:00.972 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:46:00.972 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-09-10T22:46:01.028 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:46:01.028 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T22:46:01.028 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-09-10T22:46:01.028 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-09-10T22:46:01.041 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:46:01.041 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:46:01.053 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:46:01.053 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:46:01.066 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 139934140982128 on /run/cephadm/5c4a3ee4-6fc6-11ef-bcea-c7b262605968.lock 2024-09-10T22:46:01.079 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 139934140982128 acquired on /run/cephadm/5c4a3ee4-6fc6-11ef-bcea-c7b262605968.lock 2024-09-10T22:46:01.080 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-09-10T22:46:01.080 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-09-10T22:46:01.080 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-10T22:46:01.087 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-09-10T22:46:01.087 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-09-10T22:46:01.093 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T22:46:01.093 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T22:46:01.093 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-10T22:46:01.099 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:46:01.100 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-09-10T22:46:01.100 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-09-10T22:46:01.100 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T22:46:01.100 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T22:46:01.101 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:46:11.581 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T22:46:13.399 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:46:13.400 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:46:13.400 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T22:46:14.747 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-09-10T22:46:14.747 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-09-10T22:46:15.795 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA3zOBmCiBHBxAAnclRhByr8LKng3c2vNvqGw== 2024-09-10T22:46:17.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA4zOBm2fnqFRAAMzA+EwHC/KXjiK+hR8zn+g== 2024-09-10T22:46:18.091 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA5zOBmblu2GRAANmqAePy4plkj7THKP7N+sw== 2024-09-10T22:46:18.092 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-09-10T22:46:19.204 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:46:19.204 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T22:46:19.204 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:46:19.205 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-09-10T22:46:20.258 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-09-10T22:46:20.622 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T22:46:20.846 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T22:46:21.120 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target. 2024-09-10T22:46:21.120 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target. 2024-09-10T22:46:21.445 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a 2024-09-10T22:46:21.445 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T22:46:21.722 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target.wants/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@.service. 2024-09-10T22:46:22.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:22 smithi067 podman[30562]: 2024-09-10 22:46:22.321296461 +0000 UTC m=+0.364158703 container start 7de55e7dbc7e46d99d001b00ed38f1de3a53f7e852ec1f1f7ddddf341ef61b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T22:46:22.441 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:46:22.442 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-09-10T22:46:22.455 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-09-10T22:46:22.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:22 smithi067 ceph-mon[30581]: mkfs 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:22.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:22 smithi067 ceph-mon[30581]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:46:22.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:22 smithi067 bash[30562]: 7de55e7dbc7e46d99d001b00ed38f1de3a53f7e852ec1f1f7ddddf341ef61b13 2024-09-10T22:46:22.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:22 smithi067 systemd[1]: Started Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: monmap epoch 1 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: last_changed 2024-09-10T22:46:18.509655+0000 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:46:23.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: min_mon_release 19 (squid) 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: election_strategy: 1 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: fsmap 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: mgrmap e1: no daemons active 2024-09-10T22:46:23.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:23 smithi067 ceph-mon[30581]: from='client.? 172.21.15.67:0/2850103627' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T22:46:23.561 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T22:46:23.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:23.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T22:46:23.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:23.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.616878s) 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-09-10T22:46:23.563 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T22:46:24.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:24 smithi067 ceph-mon[30581]: from='client.? 172.21.15.67:0/2194976426' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:46:24.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:24 smithi067 ceph-mon[30581]: from='client.? 172.21.15.67:0/2194976426' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:46:24.858 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:46:24.859 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-09-10T22:46:25.739 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:25 smithi067 ceph-mon[30581]: from='client.? 172.21.15.67:0/2332548967' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:26.015 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-09-10T22:46:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:26 smithi067 systemd[1]: Stopping Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:46:26.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:26 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a[30577]: 2024-09-10T22:46:26.389+0000 7fbc5b000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:46:26.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:26 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a[30577]: 2024-09-10T22:46:26.389+0000 7fbc5b000640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T22:46:26.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:26 smithi067 podman[30818]: 2024-09-10 22:46:26.615535279 +0000 UTC m=+0.368362644 container died 7de55e7dbc7e46d99d001b00ed38f1de3a53f7e852ec1f1f7ddddf341ef61b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T22:46:27.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 podman[30818]: 2024-09-10 22:46:27.058770507 +0000 UTC m=+0.811597871 container cleanup 7de55e7dbc7e46d99d001b00ed38f1de3a53f7e852ec1f1f7ddddf341ef61b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T22:46:27.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 bash[30818]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a 2024-09-10T22:46:27.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 podman[30829]: 2024-09-10 22:46:27.219448224 +0000 UTC m=+0.592759107 container remove 7de55e7dbc7e46d99d001b00ed38f1de3a53f7e852ec1f1f7ddddf341ef61b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-10T22:46:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 systemd[1]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T22:46:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 systemd[1]: Stopped Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:46:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 systemd[1]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.212s CPU time. 2024-09-10T22:46:27.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 systemd[1]: Starting Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:46:28.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 podman[30921]: 2024-09-10 22:46:27.706094456 +0000 UTC m=+0.027622181 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:46:28.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:27 smithi067 podman[30921]: 2024-09-10 22:46:27.887275014 +0000 UTC m=+0.208802729 container create f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T22:46:28.109 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 podman[30921]: 2024-09-10 22:46:28.106723338 +0000 UTC m=+0.428251064 container init f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:46:28.208 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T22:46:28.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 podman[30921]: 2024-09-10 22:46:28.113493949 +0000 UTC m=+0.435021664 container start f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:46:28.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:46:28.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:46:28.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: pidfile_write: ignore empty --pid-file 2024-09-10T22:46:28.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: load: jerasure load: lrc 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Git sha 0 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: DB SUMMARY 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: DB Session ID: 1HNSP5G776C9BWJ1A573 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:46:28.377 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95620 ; 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:46:28.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.env: 0x5575e48d2180 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.info_log: 0x5575fc2176e0 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:46:28.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.statistics: (nil) 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.use_fsync: 0 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:46:28.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.db_log_dir: 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.wal_dir: 2024-09-10T22:46:28.381 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:46:28.382 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.write_buffer_manager: 0x5575fc21bb80 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:46:28.383 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:46:28.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:46:28.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.unordered_write: 0 2024-09-10T22:46:28.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:46:28.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:46:28.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.row_cache: None 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.wal_filter: None 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:46:28.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.wal_compression: 0 2024-09-10T22:46:28.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:46:28.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:46:28.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:46:28.386 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:46:28.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:46:28.388 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_open_files: -1 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:46:28.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Compression algorithms supported: 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kZSTD supported: 0 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:46:28.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.merge_operator: 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_filter: None 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5575fc217300) 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:46:28.391 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x5575fc20b610 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-09-10T22:46:28.392 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-09-10T22:46:28.393 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-09-10T22:46:28.394 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression: NoCompression 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:46:28.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.num_levels: 7 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:46:28.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:46:28.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:46:28.398 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:46:28.399 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:46:28.400 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:46:28.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.ttl: 2592000 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:46:28.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 37471b72-53ea-42e2-9ae8-766a66f1ca55 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008388157625, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008388159783, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91777, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 287, "table_properties": {"data_size": 89818, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11801, "raw_average_key_size": 47, "raw_value_size": 83136, "raw_average_value_size": 337, "num_data_blocks": 12, "num_entries": 246, "num_filter_entries": 246, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726008388, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "37471b72-53ea-42e2-9ae8-766a66f1ca55", "db_session_id": "1HNSP5G776C9BWJ1A573", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008388160056, "job": 1, "event": "recovery_finished"} 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:46:28.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5575fc23ce00 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: DB pointer 0x5575fc33c000 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:46:28.404 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 91.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 91.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 69.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.405 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 9.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 9.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x5575fc20b610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:46:28.406 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???) e1 preinit fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).mds e1 new map 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).mds e1 print_map 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: btime 2024-09-10T22:46:22:380568+0000 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-09-10T22:46:28.407 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 bash[30921]: f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 systemd[1]: Started Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:46:28.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: monmap epoch 1 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: last_changed 2024-09-10T22:46:18.509655+0000 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: min_mon_release 19 (squid) 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: election_strategy: 1 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: fsmap 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T22:46:28.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:28 smithi067 ceph-mon[30939]: mgrmap e1: no daemons active 2024-09-10T22:46:29.494 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T22:46:29.521 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:46:29.521 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-09-10T22:46:29.522 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T22:46:29.522 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T22:46:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x 2024-09-10T22:46:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T22:46:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:29 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/261416759' entity='client.admin' 2024-09-10T22:46:30.109 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968.target.wants/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@.service. 2024-09-10T22:46:30.457 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 systemd[1]: Starting Ceph mgr.x for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:46:30.708 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 podman[31161]: 2024-09-10 22:46:30.359995822 +0000 UTC m=+0.027249994 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:46:30.708 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 podman[31161]: 2024-09-10 22:46:30.456842308 +0000 UTC m=+0.124096477 container create b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T22:46:30.708 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 podman[31161]: 2024-09-10 22:46:30.701156524 +0000 UTC m=+0.368410696 container init b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T22:46:30.708 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 podman[31161]: 2024-09-10 22:46:30.707131158 +0000 UTC m=+0.374385329 container start b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T22:46:30.819 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:46:30.819 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-10T22:46:30.834 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:46:30.834 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:46:30.834 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-10T22:46:30.835 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:46:30.856 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:46:30.856 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-09-10T22:46:30.871 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-09-10T22:46:31.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 bash[31161]: b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 2024-09-10T22:46:31.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:30 smithi067 systemd[1]: Started Ceph mgr.x for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:46:31.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:31 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1105760094' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:46:32.232 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:32.232 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:46:32.232 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:46:32.232 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:46:32.232 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:32.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:46:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:46:32.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:46:32.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:46:32.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:46:22:380568+0000", 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:46:22.381362+0000", 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:32.239 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:32.240 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:46:32.240 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:46:32.240 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-09-10T22:46:35.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:34 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1913301923' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:46:35.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:35.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:46:35.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:46:35.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:46:35.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:35.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:46:35.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:46:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:46:22:380568+0000", 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:46:35.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:46:22.381362+0000", 2024-09-10T22:46:35.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:35.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:35.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:46:35.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:46:35.474 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-09-10T22:46:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: Activating manager daemon x 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: mgrmap e2: x(active, starting, since 0.00466393s) 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: Manager daemon x is now available 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' 2024-09-10T22:46:37.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:37 smithi067 ceph-mon[30939]: from='mgr.14100 172.21.15.67:0/3900042702' entity='mgr.x' 2024-09-10T22:46:39.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:39 smithi067 ceph-mon[30939]: mgrmap e3: x(active, since 1.00971s) 2024-09-10T22:46:39.146 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:39 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3605240023' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:46:39.153 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:46:39.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:46:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:46:39.156 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:46:39.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:46:22:380568+0000", 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:46:39.160 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:46:22.381362+0000", 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:46:39.161 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:46:39.162 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:46:39.162 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-09-10T22:46:40.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:40 smithi067 ceph-mon[30939]: mgrmap e4: x(active, since 2s) 2024-09-10T22:46:40.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:40 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/409380607' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:46:40.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:40.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:46:40.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:46:40.603 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-09-10T22:46:41.760 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:41 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2201468063' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T22:46:41.760 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:41 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setuser ceph since I am not root 2024-09-10T22:46:41.760 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:41 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setgroup ceph since I am not root 2024-09-10T22:46:42.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:42 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2201468063' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T22:46:42.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:42 smithi067 ceph-mon[30939]: mgrmap e5: x(active, since 4s) 2024-09-10T22:46:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:43 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/483329385' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:46:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-09-10T22:46:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-09-10T22:46:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: Active manager daemon x restarted 2024-09-10T22:46:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: Activating manager daemon x 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: mgrmap e6: x(active, starting, since 0.00359487s) 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:46:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:47 smithi067 ceph-mon[30939]: Manager daemon x is now available 2024-09-10T22:46:49.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:49.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: Found migration_current of "None". Setting to last migration. 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:46:49.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:49 smithi067 ceph-mon[30939]: mgrmap e7: x(active, since 1.00739s) 2024-09-10T22:46:49.351 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:46:49.351 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T22:46:49.351 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:46:49.351 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:46:49.351 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-09-10T22:46:49.352 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-09-10T22:46:50.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-mon[30939]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:46:50.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-mon[30939]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:46:50.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:50.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:50.710 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:46:50.386+0000 7fbc6ea00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:46:50] ENGINE Error in HTTPServer.serve 2024-09-10T22:46:50.710 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Traceback (most recent call last): 2024-09-10T22:46:50.710 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._connections.run(self.expiration_interval) 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._run(expiration_interval) 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s = self.context.wrap_socket( 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: return self.sslsocket_class._create( 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:46:50.711 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self.do_handshake() 2024-09-10T22:46:50.712 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:46:50.712 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._sslobj.do_handshake() 2024-09-10T22:46:50.712 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:46:50.712 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:50 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:46:51.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:51.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: [10/Sep/2024:22:46:50] ENGINE Bus STARTING 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: [10/Sep/2024:22:46:50] ENGINE Serving on http://172.21.15.67:8765 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: mgrmap e8: x(active, since 2s) 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: [10/Sep/2024:22:46:50] ENGINE Serving on https://172.21.15.67:7150 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: [10/Sep/2024:22:46:50] ENGINE Bus STARTED 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:51 smithi067 ceph-mon[30939]: [10/Sep/2024:22:46:50] ENGINE Error in HTTPServer.serve 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:46:51.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-09-10T22:46:51.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:46:52.040 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T22:46:52.040 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-09-10T22:46:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:52 smithi067 ceph-mon[30939]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Generating public/private rsa key pair. 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Your identification has been saved in /tmp/tmpd1mps27s/key 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Your public key has been saved in /tmp/tmpd1mps27s/key.pub 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: The key fingerprint is: 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: SHA256:3lWUFkUCsBSKUYrjoHvVkUatAIakT3BD8yW0/P/UxIg ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: The key's randomart image is: 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: +---[RSA 3072]----+ 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: |o+Boo +o. +o..+=o| 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: |o+ =.* =.o . .o. | 2024-09-10T22:46:53.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: |. o *.*.. . .. | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: | + o =.. . o . | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: |. . o o E . + | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: | . . o . + | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: |. . o o . | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: | . o | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: | . | 2024-09-10T22:46:53.415 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:46:53 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: +----[SHA256]-----+ 2024-09-10T22:46:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:54 smithi067 ceph-mon[30939]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:54 smithi067 ceph-mon[30939]: Generating ssh key... 2024-09-10T22:46:54.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:54 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:54.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:54 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:46:54.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:46:54.993 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:46:54.993 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T22:46:55.000 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-09-10T22:46:55.258 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:55 smithi067 ceph-mon[30939]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:56.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:56 smithi067 ceph-mon[30939]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:57.979 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:46:57 smithi067 ceph-mon[30939]: Deploying cephadm binary to smithi067 2024-09-10T22:47:01.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-09-10T22:47:01.017 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-09-10T22:47:01.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:01 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:01.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:01 smithi067 ceph-mon[30939]: Added host smithi067 2024-09-10T22:47:01.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:01 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:02.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T22:47:02.601 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-09-10T22:47:02.850 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:02 smithi067 ceph-mon[30939]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:02.850 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:02 smithi067 ceph-mon[30939]: Saving service mon spec with placement count:5 2024-09-10T22:47:02.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:02 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:04.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:04 smithi067 ceph-mon[30939]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:04.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:04 smithi067 ceph-mon[30939]: Saving service mgr spec with placement count:2 2024-09-10T22:47:04.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:04 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:04.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:04 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:04.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T22:47:06.193 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:05 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:06.193 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:05 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1680812387' entity='client.admin' 2024-09-10T22:47:08.171 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-09-10T22:47:08.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:08 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:08.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:08 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2209701528' entity='client.admin' 2024-09-10T22:47:09.579 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:09 smithi067 ceph-mon[30939]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:09.579 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:09 smithi067 ceph-mon[30939]: from='mgr.14118 172.21.15.67:0/3296254112' entity='mgr.x' 2024-09-10T22:47:09.579 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:09 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4038773102' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T22:47:10.215 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:09 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setuser ceph since I am not root 2024-09-10T22:47:10.215 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:09 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setgroup ceph since I am not root 2024-09-10T22:47:10.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:10 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4038773102' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T22:47:10.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:10 smithi067 ceph-mon[30939]: mgrmap e9: x(active, since 22s) 2024-09-10T22:47:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:11 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3254427025' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:47:12.635 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:47:12.635 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-09-10T22:47:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-09-10T22:47:16.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: Active manager daemon x restarted 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: Activating manager daemon x 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: mgrmap e10: x(active, starting, since 0.00417141s) 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:47:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:16 smithi067 ceph-mon[30939]: Manager daemon x is now available 2024-09-10T22:47:17.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:17 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:17.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:17 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:47:17.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:17 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:47:17.475 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:17 smithi067 ceph-mon[30939]: mgrmap e11: x(active, since 1.00776s) 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi067.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T22:47:18.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-mon[30939]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:47:18.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-mon[30939]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:47:18.722+0000 7fe6cfa00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:47:18] ENGINE Error in HTTPServer.serve 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Traceback (most recent call last): 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._connections.run(self.expiration_interval) 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._run(expiration_interval) 2024-09-10T22:47:19.049 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s = self.context.wrap_socket( 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: return self.sslsocket_class._create( 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self.do_handshake() 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._sslobj.do_handshake() 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:47:19.050 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:47:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: mgrmap e12: x(active, since 2s) 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: [10/Sep/2024:22:47:18] ENGINE Bus STARTING 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: [10/Sep/2024:22:47:18] ENGINE Serving on https://172.21.15.67:7150 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: [10/Sep/2024:22:47:18] ENGINE Error in HTTPServer.serve 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:47:19.549 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:47:19.550 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: [10/Sep/2024:22:47:18] ENGINE Serving on http://172.21.15.67:8765 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: [10/Sep/2024:22:47:18] ENGINE Bus STARTED 2024-09-10T22:47:19.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:19 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:21.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:21.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUWi6elqsxYYnIIrkAod2wkXJkGtYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIyNDcyMFoXDTM0\nMDkxMTIyNDcyMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjY3MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAjy//u+jd5Gut0wihTxPSMHK/gNscjoU/RSMm\nT4/b/bK+24yMhsnKilllcII3oxQw52aTQ/h7qGV99VBpYgc+C49F1mC0eCsrntYc\niCLjcVn2MTqttDPqNS76yvppzfpVYbqF5sdw9+NFa78Hdbcr7mC6yndQxRe8mNV7\nXzyRfeHGWt3AkRNLqGe4vJIpU6vnooYr2okr0jfOtbEWiyzcu5fmIwbGPpufWKyZ\nWBz7YaA3mlog06ufh2IvmzW0/yhcbMKhAu4Qs0TAjsqwj2EnUeULSOEFIrEIQVZO\ncH2Aa2CuJ52rcyBi2eYy3y7ewDEcgFZif3HYyveP5cS8LKMvk9VYfykYFNYz+wT1\n/LiEsSc9BEMDkHpMlJ4Gf9b/cqv9lngFjFqx+1DssZpgw6QDVk4Bmx1HfrqSStFN\nMkCYL46GW9w/ZZE+y9Ff0rwUQjzijQlcpiWdG562PD1VDGzctLnINk2pzX4Pjnov\n+JY320n0lC0quzyoTaa9IQSQLMj/2TU/eBAfMnh9ppjLhlbogBE1kY1jWQvx70u6\ntcIIBuoN/yfUso9hI14b8hA0fmlNU6AsGoKEoEpEy0be8TcRcNV4cVfwjrOUV7Bp\nCqz1JyNiMx+KX9ery7VKmCKwIfDuebfTRgoZ3MWjtaHDRahaSaIlpD2S78kY0FiO\nVxzaxHcCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDY3ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPQzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQB+\nnKfXpJI9AqxexhnluhsMrRj4NkfEysYi7wg/7ubMmQIyMZ5/E8PYLu/8Ldekt9I5\n3x7jbWTxIaBTYvBOfufEWf8dtQE07BIAh7/Fg97ME3SLro/pZvTfFroseKRJVRl1\nyFpHyFnhhfwMR1JDOKWVd4cDlt6ohiI7mjxVQGVUwNxFQ3quDGRVkva3yMlJhSSs\nJz5fudOcLxDiA7rQ0yLQEEN7G+DHakHmBuSeoeNN5PK5zzjd0RwRe9pDBvpovsH1\nCCGfuQfYDU9/TrfO+K74xWjkLXPQjkU/iP3aL0qJhtxKBHqr72x10PSr28X/42/j\nsczBzCn75KYI5sSjCUR83xGGNqVuivCkR5ubnc/vw7HrnO8bwn+WFRVTqDtnP4dH\nY8OAaBGBunslF0wtWQFj2NVQm8UXZc+MOrzV1FjTD9q5x6rFHYvCDME9uNSyJnsC\n9sQL09WE3Hr0DjGVJ4ac66/5gmS3AyXbq9IjCm/C8HARUN/m79DRthtLZqWANNLJ\nUqA3ZcBJX5XebdvVieyhm6x1VNlWpA8ZJOj1S2gco3/mvIXvKPGvA2g9F9np7ziq\nJe9T5WZjm9INVQdTAQ9Z8mBk1kQfRcWeupNwPA6JVBOvfsYVciB0B9kx9zO7Vvf0\nzKLOUBz+Mf4j+0QVYhNmIAr3M+MPeR9q6XFQSIM0Hw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAjy//u+jd5Gut0wihTxPSMHK/gNscjoU/RSMmT4/b/bK+24yM\nhsnKilllcII3oxQw52aTQ/h7qGV99VBpYgc+C49F1mC0eCsrntYciCLjcVn2MTqt\ntDPqNS76yvppzfpVYbqF5sdw9+NFa78Hdbcr7mC6yndQxRe8mNV7XzyRfeHGWt3A\nkRNLqGe4vJIpU6vnooYr2okr0jfOtbEWiyzcu5fmIwbGPpufWKyZWBz7YaA3mlog\n06ufh2IvmzW0/yhcbMKhAu4Qs0TAjsqwj2EnUeULSOEFIrEIQVZOcH2Aa2CuJ52r\ncyBi2eYy3y7ewDEcgFZif3HYyveP5cS8LKMvk9VYfykYFNYz+wT1/LiEsSc9BEMD\nkHpMlJ4Gf9b/cqv9lngFjFqx+1DssZpgw6QDVk4Bmx1HfrqSStFNMkCYL46GW9w/\nZZE+y9Ff0rwUQjzijQlcpiWdG562PD1VDGzctLnINk2pzX4Pjnov+JY320n0lC0q\nuzyoTaa9IQSQLMj/2TU/eBAfMnh9ppjLhlbogBE1kY1jWQvx70u6tcIIBuoN/yfU\nso9hI14b8hA0fmlNU6AsGoKEoEpEy0be8TcRcNV4cVfwjrOUV7BpCqz1JyNiMx+K\nX9ery7VKmCKwIfDuebfTRgoZ3MWjtaHDRahaSaIlpD2S78kY0FiOVxzaxHcCAwEA\nAQKCAgAVDC+xzgGr/X4QKtEXYCkB8SQgS1UPyBtsTMXpZRKn4Ln4aNkRSSFKI+jq\n2GFYSfQEV8om8CQxNTYGNNtAyyK+LUHNSOrOTT48u2l3Q3WeaGRtiRwWhJkVV0IL\nSzL5UaZyVqovKbr73T8Lnwl4LsQ45qQeddp8FkilSKA5eBxE3ka7faQ8TxNMfm6j\n3XJQt9C1TbatiYNVA9oRUCYpsB8EPkhYdrr2YgbS0JA76Qq0p4n02PHBW5V4H6Ey\n1fq0lRi87gEf/qGa0GzjsHyynsvaWGpHvLf2CQLXI9ok4/XYEt6WuTEGG8kcEaAB\nuZCJvM5dORWCeVguQaS32Cr2/sfT4W2u1Zty+rNDZeKB/iSGhtNEWDCSpqK+Q4le\n+v3J2dhfq3AAOIY1uZ1uLCW+PKhC4uFPG63m+bzrlq9ZsOm9RB1q8BeoBMdgL/17\n3//KmOguGZ1Vf+vb36NPQ53gj9JREiLxBXQ8Xnt7/OiXEpk9PMgUYV67Nve8NcTn\nf0S044rXqFy1mbVCuirq6x6tUtDShN6IkmOtB1LPm2F0P4YzAwgV/kfVOAbmv1Xr\nGLJ/ogAfBSG6g1fKBhXLPJExuTo7VlRGjvBnqaCAjlXfXFGvRq/zGpPm9AVJWew+\n+dFPvqALEiLmc+Aj+Gu0ZrGHsyL7vmXbeHGKw/PM4+zfiuaRCQKCAQEAyA0w3JnU\n9AMMpBlY9SUB0TBiyCICcbjGPyYIMRDww2umrTY5IPD0bx0UncfRvAiflnyDjDVg\nCfRyZJ9Kf+uBHPN3ug08NA98e8yOo3g/7ax6c5EOSfYLt6lHxVyJj4/3HI5BuR1s\n2CqMRME7EcnySVQ9GVh2Sigm3V2u08BBV+sPCApvzL1tfg5yP/0HmiZ/cdonh8OO\n3sOLbnVOhVhZo6vZ1ovHRLRqOiuUsm8GQdMqpiKDrvaXL7P45MIJfzrMBbNKTx66\nRdxL5Ws3b9tWPxokyEDIFji0AdFnOF8Pv4jqwCeyle4qUvNfjgwG0P0bBP7jcrgY\nQYp4XnyPmN2tnwKCAQEAtzuX+wlZN8QbuQzLgCDA2jGXMnFWWwabszE9B5w9HkiG\nmrTd1abfGwVCJELjaakC0iRL++yetMPPlf14j29UquAnSR/PL+RwUOuWis51ArKO\n2a4eolzpuqrR7Kqa3mR8wwA7jHnwnqucFoMA4JDwYEwff9dovLCQgsoX7X64Bx/y\nC5uinr8ttUIzAQD5SIGa2j5YBSp0aLzXbeXrcWWkEwAFVt63ZYLcqdcwDf5vYoSJ\nYVZHFuKY7N0G/XPjH3b2bovgFvHV3NwrIL9bn9Jh+bviUg1gHKUbrnz7BYwd37bN\nlYv2xDxIhc3Z8IijO7BjZJKiK9YKLfk6a63m7pZKKQKCAQEAlAo+6GDmWBSRnDYt\nwGu8ZoN59kRErsdf3YdVdInmwZa+Z1vC9v+Yw0kv4nrmWFHBV2mZJ36wLW254YCH\nbxZmG3iIVyRIMvZq1mpP++SW2llS3IaxnkX38dfRhtmt6pFM+tLDDCwcSfNijda4\n0bcyFtXBySnPj50a9aHIKGcTc+4812Rxy386sZgbMAMdRxPKKASaUDuFVJA8dGdW\nzhLNcvNPZxdVRSCFo7pwdYL3jGqqGvtbm3Rb+S8jtpVo1cA7DtTY3qgMjHozp/ip\ns5sKvfOo2JAoAVnAy7+ctPiT/NdjcQNvgsgWhgAX7FQ2Dx3eitnckx5sxMOlw7ZF\nxYf2XQKCAQEAheFQKcdp3do3W11i74p+/q5PSDcRns7UQvjOG9pnudHKMA3JLIuR\n5n9QFYid+xzXh82nizoXuAJ+kZwzU7JHzdacPcy1KMwj2KxaQa+aRU9MGvcjQQTc\nblzTp7X8FFSd6CkqWXjbfEPtd3Fe2bZtPB9CwLkWuQ/VdwrMsuXs9p+7zgl4p78O\nWas18xzQB89S4vW8gIHFrqWUTqM5BZD7cjqIrDFtFQmpy5sEbKUko3otmuZcaPdG\nP4M70UK0h8/3PUyMfpdS6N4h0PqgqsUge0VxlvxY5nUpJbQCAcqgJU6/Je6vbjCR\n2NpQzHQ5uKon5VtSjO8GXxMEQZs96cP8CQKCAQAmqbbjtEhMl72MO3C8uFvmOKAQ\nn1qx8XdINBc6Gt1nLuRGJWGnmyUBYR2Mj5WKVOYKGQ6A3PMtfGyYOJxOeLzyT9O9\nsvb5uSmz6po1DuP36S76DYtj7ziToOhnh/rySDZDEc42GLyD+DGhv4DCn1LJmxvN\n0mi7weTui5fhtq5uW69dctF1yXgS63bQuVxLVpO6Bw51hgNFETeKRmzViU9GYDCF\n0o9g1+zPNBMtPY09PZr+zJsnBZ+XPrSi2iu6GwuS2FLHAkJ5+Wwp7umnffz5qmVB\nK9Xd2dahH24zf0EhsVW5ZWPcLet57Zbw0Npb0lMD+QxOmpHqBYtogKs5kfVg\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T22:47:23.669 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T22:47:23.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:23 smithi067 ceph-mon[30939]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:23.936 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:23 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:25.051 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T22:47:25.051 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-09-10T22:47:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:25 smithi067 ceph-mon[30939]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:26.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:26 smithi067 ceph-mon[30939]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:26.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:26 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:26.703 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$StA.jBlpN/T6CI8C2ojhrug7tjvrbXzX0qThBkMlCr0mw0Po3blEq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726008446, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T22:47:26.703 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-09-10T22:47:27.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:27 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3821860101' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T22:47:28.037 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T22:47:28.055 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:47:28.055 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-09-10T22:47:28.069 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:47:28.069 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-09-10T22:47:28.069 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-09-10T22:47:28.069 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout: Password: l7w50hf9bu 2024-09-10T22:47:28.071 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:28.072 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config directory 2024-09-10T22:47:29.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:29 smithi067 ceph-mon[30939]: mgrmap e13: x(active, since 12s) 2024-09-10T22:47:29.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:29 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2807956791' entity='client.admin' 2024-09-10T22:47:29.585 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T22:47:29.585 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T22:47:29.585 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.585 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T22:47:29.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:29.587 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-09-10T22:47:29.647 INFO:tasks.cephadm:Fetching config... 2024-09-10T22:47:29.647 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:47:29.647 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T22:47:29.685 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T22:47:29.685 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:47:29.685 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T22:47:29.761 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T22:47:29.761 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:47:29.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T22:47:29.859 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T22:47:29.860 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:47:29.860 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T22:47:29.936 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T22:47:29.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:47:30.057 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:47:30.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:47:30.179 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:47:30.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:47:30.302 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ6QB0lvazZVvOtZFcRpZG5pS1ZbayaAHFYTydfjDSH3vTbuo1G+ICSuUe1In2f01kQXcw0PiAIx2wmt5JSN6dZPmDHRS/60okVYexzT8KJaJVBdi+zFmSv83wlO/aIQ6unYtchuMC4/Q2jlLYmgO5/eUxGgr6qr545bz+jUvq7kHsEID+CY9wHfLFdiM18rBBAkHXusLBZgFkirRlBilh5ZH/AZjyLt/X5202b1go9yyHWzhsZdoGRgj1e2qX75JFppdBTlKc8D9LQoLMKpqu0/H7dCeGgfaQdTbn04tL5ZMvObi1MoC5pdvfhu15LTwGw+cO7A6452b7HnTVt8L8cX9QQz8ucsC7RgqbxClx93/ftmOQ+3jH7WdRid9BfD74udLXg8wXr3HNSlkcSTjFn++HWgHs/gJrbgR0lVPvJlj/vQA+4ZbSK/J+hrXblUTS/2MGdxkIo0oPlQTfpDswH3pYqdi7l7ds4ii8q54NHbMfO936V68MCt5DsgKG+Rc= ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:47:30.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T22:47:30.733 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:32.688 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T22:47:32.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T22:47:32.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:32 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2461867315' entity='client.admin' 2024-09-10T22:47:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:34.915 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-09-10T22:47:34.916 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:47:34.916 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:47:34.957 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:47:34.957 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:35.038 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-09-10T22:47:35.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch host add smithi092 2024-09-10T22:47:35.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:35 smithi067 ceph-mon[30939]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:35.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:35 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:35.448 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:36.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:36.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:36.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:36.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:47:36.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:36 smithi067 ceph-mon[30939]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:47:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:47:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:37 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:38.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:38 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:38.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:38 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:38.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:38 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:38.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:38 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:38.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:38 smithi067 ceph-mon[30939]: Deploying cephadm binary to smithi092 2024-09-10T22:47:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:39 smithi067 ceph-mon[30939]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:40.789 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-09-10T22:47:41.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:47:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:42.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:42.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: Added host smithi092 2024-09-10T22:47:42.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:42.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:42.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:41 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:42.535 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:42.535 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-09-10T22:47:43.344 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-09-10T22:47:43.345 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:47:43.345 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:47:43.383 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:47:43.383 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:43.461 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-09-10T22:47:43.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch host add smithi145 2024-09-10T22:47:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:43.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:43 smithi067 ceph-mon[30939]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:43.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:43 smithi067 ceph-mon[30939]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:43.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:43 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:45 smithi067 ceph-mon[30939]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:45 smithi067 ceph-mon[30939]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: Deploying cephadm binary to smithi145 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:47 smithi067 ceph-mon[30939]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:47:49.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:48 smithi067 ceph-mon[30939]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:47:49.081 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-09-10T22:47:49.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:47:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:49 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:50.807 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:47:50.807 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-09-10T22:47:51.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:50 smithi067 ceph-mon[30939]: Added host smithi145 2024-09-10T22:47:51.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:50 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:51.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:50 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:51.482 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T22:47:51.482 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T22:47:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:51 smithi067 ceph-mon[30939]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:51 smithi067 ceph-mon[30939]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:53.114 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-09-10T22:47:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:53 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:53.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:53 smithi067 ceph-mon[30939]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:53.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:53 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:47:53.944 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-09-10T22:47:53.944 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-09-10T22:47:53.944 INFO:tasks.cephadm:Adding mon.c on smithi145 2024-09-10T22:47:53.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c' 2024-09-10T22:47:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:54 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:47:54.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:54 smithi067 ceph-mon[30939]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:47:54.306 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T22:47:54.399 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T22:47:55.233 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mon update... 2024-09-10T22:47:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:55 smithi067 ceph-mon[30939]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:56.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:56.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c;count:3 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:56.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:56 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:56.573 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T22:47:56.578 DEBUG:teuthology.orchestra.run.smithi145:mon.c> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T22:47:56.582 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:47:56.582 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:47:56.986 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:47:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:57 smithi067 ceph-mon[30939]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:47:57.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:57 smithi067 ceph-mon[30939]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:47:57.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:57 smithi067 ceph-mon[30939]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:47:57.838 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:47:57.838 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","modified":"2024-09-10T22:46:18.509655Z","created":"2024-09-10T22:46:18.509655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T22:47:57.839 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-09-10T22:47:58.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/576050679' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:58.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:58 smithi067 ceph-mon[30939]: Deploying daemon mon.c on smithi145 2024-09-10T22:47:59.565 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:47:59.565 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:48:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:47:59 smithi067 ceph-mon[30939]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:00.825 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:48:02.082 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:48:02.083 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","modified":"2024-09-10T22:46:18.509655Z","created":"2024-09-10T22:46:18.509655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T22:48:02.083 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-09-10T22:48:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:01 smithi067 ceph-mon[30939]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 podman[34379]: 2024-09-10 22:48:02.974073642 +0000 UTC m=+0.027742942 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:48:03.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:02 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/146991041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:03.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 podman[34379]: 2024-09-10 22:48:03.270063972 +0000 UTC m=+0.323733272 container init 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T22:48:03.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 podman[34379]: 2024-09-10 22:48:03.276485583 +0000 UTC m=+0.330154886 container start 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T22:48:03.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:48:03.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:48:03.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pidfile_write: ignore empty --pid-file 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: load: jerasure load: lrc 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Git sha 0 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: DB SUMMARY 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: DB Session ID: UK5KKT3C3JRIJ4TYUBL3 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T22:48:03.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.env: 0x560e5190a180 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:48:03.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.info_log: 0x560e6ea396e0 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.statistics: (nil) 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.use_fsync: 0 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:48:03.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.db_log_dir: 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.wal_dir: 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:48:03.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.write_buffer_manager: 0x560e6ea3db80 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:48:03.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.unordered_write: 0 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.row_cache: None 2024-09-10T22:48:03.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.wal_filter: None 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.wal_compression: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:48:03.639 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:48:03.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:48:03.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_open_files: -1 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Compression algorithms supported: 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kZSTD supported: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:48:03.642 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.merge_operator: 2024-09-10T22:48:03.643 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_filter: None 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e6ea39300) 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:48:03.644 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_type: 0 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_index_type: 0 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_shortening: 1 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: checksum: 4 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: no_block_cache: 0 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache: 0x560e6ea2d610 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_options: 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: capacity : 536870912 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_shard_bits : 4 2024-09-10T22:48:03.645 INFO:journalctl@ceph.mon.c.smithi145.stdout: strict_capacity_limit : 0 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_compressed: (nil) 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: persistent_cache: (nil) 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size: 4096 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size_deviation: 10 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_restart_interval: 16 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_block_restart_interval: 1 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: metadata_block_size: 4096 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: partition_filters: 0 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: use_delta_encoding: 1 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: filter_policy: bloomfilter 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: whole_key_filtering: 1 2024-09-10T22:48:03.646 INFO:journalctl@ceph.mon.c.smithi145.stdout: verify_compression: 0 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: format_version: 5 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_index_compression: 1 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_align: 0 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: max_auto_readahead_size: 262144 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: prepopulate_block_cache: 0 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression: NoCompression 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:48:03.647 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.num_levels: 7 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:48:03.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:48:03.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:48:03.650 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:48:03.651 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:48:03.652 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.ttl: 2592000 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:48:03.653 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 61f743b9-881b-4aa4-ac8f-10eb0beb904e 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008483319724, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008483320451, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726008483, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "61f743b9-881b-4aa4-ac8f-10eb0beb904e", "db_session_id": "UK5KKT3C3JRIJ4TYUBL3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008483320728, "job": 1, "event": "recovery_finished"} 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e6ea5ee00 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: DB pointer 0x560e6eb5e000 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:48:03.654 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** DB Stats ** 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:48:03.655 INFO:journalctl@ceph.mon.c.smithi145.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:48:03.656 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache BinnedLRUCache@0x560e6ea2d610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: using public_addr v2:172.21.15.145:0/0 -> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: starting mon.c rank -1 at public addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] at bind addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(???) e0 preinit fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:03.657 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: e1 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: btime 2024-09-10T22:46:22:380568+0000 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: legacy client fscid: -1 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout: No filesystems configured 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T22:48:03.658 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Deploying cephadm binary to smithi145 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:03.659 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.660 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Added host smithi145 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:48:03.661 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c;count:3 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:03.662 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/576050679' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: Deploying daemon mon.c on smithi145 2024-09-10T22:48:03.663 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.664 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:03.664 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/146991041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:03.664 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 ceph-mon[34397]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T22:48:03.664 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 bash[34379]: 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 2024-09-10T22:48:03.664 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:03 smithi145 systemd[1]: Started Ceph mon.c for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:48:03.890 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:48:03.891 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:48:04.256 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: mon.a calling monitor election 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: mon.c calling monitor election 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: monmap epoch 2 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: last_changed 2024-09-10T22:48:03.360881+0000 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: min_mon_release 19 (squid) 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: election_strategy: 1 2024-09-10T22:48:08.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: fsmap 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: mgrmap e13: x(active, since 52s) 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:08.801 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:08 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: mon.a calling monitor election 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: mon.c calling monitor election 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:08.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: monmap epoch 2 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: last_changed 2024-09-10T22:48:03.360881+0000 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: min_mon_release 19 (squid) 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: election_strategy: 1 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: fsmap 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: mgrmap e13: x(active, since 52s) 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.883 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:08.883 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:08.883 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:08 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:09.439 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:48:09.439 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","modified":"2024-09-10T22:48:03.360881Z","created":"2024-09-10T22:46:18.509655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:48:09.439 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-09-10T22:48:09.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:09 smithi145 ceph-mon[34397]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:09.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:09 smithi145 ceph-mon[34397]: Deploying daemon mon.b on smithi092 2024-09-10T22:48:09.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:09 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:09.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:09 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/4000347573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:09 smithi067 ceph-mon[30939]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:09 smithi067 ceph-mon[30939]: Deploying daemon mon.b on smithi092 2024-09-10T22:48:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:09 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:09.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:09 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/4000347573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:10.798 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:10 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:48:10.361+0000 7fe710000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T22:48:11.134 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:48:11.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:48:11.494 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:48:11.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:11 smithi145 ceph-mon[34397]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:11 smithi067 ceph-mon[30939]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:12.255 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:48:12.255 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","modified":"2024-09-10T22:48:03.360881Z","created":"2024-09-10T22:46:18.509655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:48:12.255 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-09-10T22:48:12.588 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:12 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/29505584' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:12 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/29505584' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:13.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 podman[34205]: 2024-09-10 22:48:13.279068185 +0000 UTC m=+0.133615417 container create 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:48:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:13 smithi067 ceph-mon[30939]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:13 smithi145 ceph-mon[34397]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:13.911 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:48:13.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:48:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 podman[34205]: 2024-09-10 22:48:13.525831699 +0000 UTC m=+0.380378936 container init 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 podman[34205]: 2024-09-10 22:48:13.532457211 +0000 UTC m=+0.387004438 container start 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pidfile_write: ignore empty --pid-file 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: load: jerasure load: lrc 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Git sha 0 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:48:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: DB SUMMARY 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: DB Session ID: AMLGTVNF5119C0RSZPY6 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:48:14.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:48:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.env: 0x55f05006f180 2024-09-10T22:48:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:48:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.info_log: 0x55f08d5fd6e0 2024-09-10T22:48:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.statistics: (nil) 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.use_fsync: 0 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:48:14.026 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.db_log_dir: 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.wal_dir: 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:48:14.027 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.write_buffer_manager: 0x55f08d601b80 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:48:14.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.unordered_write: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.row_cache: None 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.wal_filter: None 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.wal_compression: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:48:14.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:48:14.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_open_files: -1 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:48:14.031 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Compression algorithms supported: 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kZSTD supported: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:48:14.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.merge_operator: 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_filter: None 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f08d5fd300) 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:48:14.033 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_type: 0 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_index_type: 0 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_shortening: 1 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: checksum: 4 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: no_block_cache: 0 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache: 0x55f08d5f1610 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_options: 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: capacity : 536870912 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_shard_bits : 4 2024-09-10T22:48:14.034 INFO:journalctl@ceph.mon.b.smithi092.stdout: strict_capacity_limit : 0 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_compressed: (nil) 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: persistent_cache: (nil) 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size: 4096 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size_deviation: 10 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_restart_interval: 16 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_block_restart_interval: 1 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: metadata_block_size: 4096 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: partition_filters: 0 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: use_delta_encoding: 1 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: filter_policy: bloomfilter 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: whole_key_filtering: 1 2024-09-10T22:48:14.035 INFO:journalctl@ceph.mon.b.smithi092.stdout: verify_compression: 0 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: format_version: 5 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_index_compression: 1 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_align: 0 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: max_auto_readahead_size: 262144 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: prepopulate_block_cache: 0 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression: NoCompression 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:48:14.036 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.num_levels: 7 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:48:14.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:48:14.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:48:14.039 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:48:14.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:48:14.041 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.ttl: 2592000 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:48:14.042 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 18cf33ac-6e90-4db9-ad9b-85f7c3c49cf4 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008493575331, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008493576221, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726008493, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "18cf33ac-6e90-4db9-ad9b-85f7c3c49cf4", "db_session_id": "AMLGTVNF5119C0RSZPY6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008493576479, "job": 1, "event": "recovery_finished"} 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f08d622e00 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: DB pointer 0x55f08d722000 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:48:14.043 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** DB Stats ** 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:14.044 INFO:journalctl@ceph.mon.b.smithi092.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:48:14.045 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval compaction: 0.00 GB write, 0.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache BinnedLRUCache@0x55f08d5f1610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: using public_addr v2:172.21.15.92:0/0 -> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: starting mon.b rank -1 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:14.046 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(???) e0 preinit fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 bash[34205]: 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: e1 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: btime 2024-09-10T22:46:22:380568+0000 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: legacy client fscid: -1 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout: No filesystems configured 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T22:48:14.047 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Deploying cephadm binary to smithi145 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.048 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:14.049 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Added host smithi145 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3118622245' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c;count:3 2024-09-10T22:48:14.050 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:14.051 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/576050679' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Deploying daemon mon.c on smithi145 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/146991041' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.052 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.a calling monitor election 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.c calling monitor election 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: monmap epoch 2 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: last_changed 2024-09-10T22:48:03.360881+0000 2024-09-10T22:48:14.053 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: min_mon_release 19 (squid) 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: election_strategy: 1 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: fsmap 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mgrmap e13: x(active, since 52s) 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:14.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: Deploying daemon mon.b on smithi092 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/4000347573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/29505584' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 systemd[1]: Started Ceph mon.b for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:48:14.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:13 smithi092 ceph-mon[34223]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-10T22:48:14.265 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: mon.a calling monitor election 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: mon.c calling monitor election 2024-09-10T22:48:19.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: mon.b calling monitor election 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: monmap epoch 3 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: last_changed 2024-09-10T22:48:13.620672+0000 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:19.023 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: min_mon_release 19 (squid) 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: election_strategy: 1 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: fsmap 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: mgrmap e13: x(active, since 62s) 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:18 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: mon.a calling monitor election 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: mon.c calling monitor election 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: mon.b calling monitor election 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: monmap epoch 3 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: last_changed 2024-09-10T22:48:13.620672+0000 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: min_mon_release 19 (squid) 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: election_strategy: 1 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: fsmap 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:19.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: mgrmap e13: x(active, since 62s) 2024-09-10T22:48:19.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T22:48:19.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:18 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: mon.a calling monitor election 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: mon.c calling monitor election 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: mon.b calling monitor election 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: monmap epoch 3 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: last_changed 2024-09-10T22:48:13.620672+0000 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: created 2024-09-10T22:46:18.509655+0000 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: min_mon_release 19 (squid) 2024-09-10T22:48:19.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: election_strategy: 1 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: fsmap 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: mgrmap e13: x(active, since 62s) 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:18 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:19.465 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:48:19.465 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","modified":"2024-09-10T22:48:13.620672Z","created":"2024-09-10T22:46:18.509655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T22:48:19.466 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 3 2024-09-10T22:48:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:19 smithi145 ceph-mon[34397]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:19 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/3827574625' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:19.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:19 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:19 smithi092 ceph-mon[34223]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:19 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/3827574625' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:19 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:19 smithi067 ceph-mon[30939]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:19 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/3827574625' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:48:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:19 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:20.945 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T22:48:20.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T22:48:20.995 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:20 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:48:20.620+0000 7fe710000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T22:48:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:21.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:21 smithi067 ceph-mon[30939]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:21.341 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:21.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:21.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:21.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:21.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:21 smithi145 ceph-mon[34397]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:21.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:21 smithi092 ceph-mon[34223]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:22.173 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-09-10T22:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:48:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: Reconfiguring daemon mon.a on smithi067 2024-09-10T22:48:22.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:22 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2918818643' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:22.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: Reconfiguring daemon mon.a on smithi067 2024-09-10T22:48:22.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:22 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2918818643' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:22.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: Reconfiguring daemon mon.a on smithi067 2024-09-10T22:48:22.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:22 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2918818643' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:23.223 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T22:48:23.224 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:48:23.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:48:23.282 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:48:23.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:23.378 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:48:23.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:48:23.441 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:48:23.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:23.538 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:48:23.539 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:48:23.602 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:48:23.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:23.699 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-10T22:48:23.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-10T22:48:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:23 smithi067 ceph-mon[30939]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:23 smithi145 ceph-mon[34397]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:23 smithi092 ceph-mon[34223]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:24.302 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:25.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:48:25.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:25 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T22:48:25.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:25.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:48:25.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T22:48:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='mgr.14150 172.21.15.67:0/1928849452' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:48:25.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:25 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T22:48:26.048 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setuser ceph since I am not root 2024-09-10T22:48:26.048 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:25 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ignoring --setgroup ceph since I am not root 2024-09-10T22:48:26.461 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-09-10T22:48:26.461 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-09-10T22:48:26.461 INFO:tasks.cephadm:Adding mgr.z on smithi145 2024-09-10T22:48:26.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi067=x;smithi092=y;smithi145=z' 2024-09-10T22:48:26.823 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:48:26.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:26 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T22:48:26.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:26 smithi145 ceph-mon[34397]: mgrmap e14: x(active, since 69s) 2024-09-10T22:48:27.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:26 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T22:48:27.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:26 smithi092 ceph-mon[34223]: mgrmap e14: x(active, since 69s) 2024-09-10T22:48:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:26 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3547331330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T22:48:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:26 smithi067 ceph-mon[30939]: mgrmap e14: x(active, since 69s) 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: Active manager daemon x restarted 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: Activating manager daemon x 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: mgrmap e15: x(active, starting, since 0.00615516s) 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:48:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:48:32.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:48:32.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:48:32.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:32 smithi092 ceph-mon[34223]: Manager daemon x is now available 2024-09-10T22:48:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: Active manager daemon x restarted 2024-09-10T22:48:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: Activating manager daemon x 2024-09-10T22:48:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T22:48:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: mgrmap e15: x(active, starting, since 0.00615516s) 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:48:32.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:32 smithi067 ceph-mon[30939]: Manager daemon x is now available 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: Active manager daemon x restarted 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: Activating manager daemon x 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: mgrmap e15: x(active, starting, since 0.00615516s) 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:48:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:48:32.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:48:32.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:48:32.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:32 smithi145 ceph-mon[34397]: Manager daemon x is now available 2024-09-10T22:48:32.984 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mgr update... 2024-09-10T22:48:33.265 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:33.265 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:48:33.266 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:48:33.266 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:33 smithi145 ceph-mon[34397]: mgrmap e16: x(active, since 1.01338s) 2024-09-10T22:48:33.266 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:48:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:48:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:33 smithi092 ceph-mon[34223]: mgrmap e16: x(active, since 1.01338s) 2024-09-10T22:48:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:48:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:48:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:33 smithi067 ceph-mon[30939]: mgrmap e16: x(active, since 1.01338s) 2024-09-10T22:48:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:33.632 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T22:48:33.637 DEBUG:teuthology.orchestra.run.smithi145:mgr.z> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T22:48:33.642 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T22:48:33.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T22:48:33.646 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T22:48:33.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T22:48:33.692 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T22:48:33.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T22:48:33.739 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T22:48:33.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T22:48:33.786 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T22:48:33.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T22:48:33.834 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T22:48:33.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T22:48:33.890 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T22:48:33.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T22:48:33.948 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T22:48:33.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T22:48:34.013 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T22:48:34.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T22:48:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:34 smithi092 ceph-mon[34223]: Saving service mgr spec with placement smithi067=x;smithi092=y;smithi145=z;count:3 2024-09-10T22:48:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.277 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T22:48:34.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T22:48:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-mon[30939]: Saving service mgr spec with placement smithi067=x;smithi092=y;smithi145=z;count:3 2024-09-10T22:48:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.310 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:34 smithi145 ceph-mon[34397]: Saving service mgr spec with placement smithi067=x;smithi092=y;smithi145=z;count:3 2024-09-10T22:48:34.310 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.310 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:34.312 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T22:48:34.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T22:48:34.380 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T22:48:34.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T22:48:34.437 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T22:48:34.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T22:48:34.493 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T22:48:34.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T22:48:34.562 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T22:48:34.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T22:48:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:34.586 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T22:48:34.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T22:48:34.653 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T22:48:34.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T22:48:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:34.702 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T22:48:34.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:48:34.708+0000 7fc9fda00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:48:34] ENGINE Error in HTTPServer.serve 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: Traceback (most recent call last): 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._connections.run(self.expiration_interval) 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._run(expiration_interval) 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:48:34.724 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: s = self.context.wrap_socket( 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: return self.sslsocket_class._create( 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self.do_handshake() 2024-09-10T22:48:34.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:48:34.726 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: self._sslobj.do_handshake() 2024-09-10T22:48:34.726 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:48:34.726 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:48:34 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:48:34.729 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T22:48:34.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T22:48:34.739 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:34.744 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-10T22:48:34.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-09-10T22:48:34.800 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-10T22:48:34.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-10T22:48:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:34.835 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-10T22:48:34.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-10T22:48:34.889 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-10T22:48:34.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-10T22:48:34.946 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T22:48:34.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T22:48:35.003 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T22:48:35.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T22:48:35.056 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T22:48:35.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T22:48:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:35.109 DEBUG:tasks.cephadm:set 26 configs 2024-09-10T22:48:35.109 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi067') 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: mgrmap e17: x(active, since 2s) 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: [10/Sep/2024:22:48:34] ENGINE Bus STARTING 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: [10/Sep/2024:22:48:34] ENGINE Serving on http://172.21.15.67:8765 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: [10/Sep/2024:22:48:34] ENGINE Serving on https://172.21.15.67:7150 2024-09-10T22:48:35.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: [10/Sep/2024:22:48:34] ENGINE Bus STARTED 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: [10/Sep/2024:22:48:34] ENGINE Error in HTTPServer.serve 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:48:35.281 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-09-10T22:48:35.282 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: mgrmap e17: x(active, since 2s) 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: [10/Sep/2024:22:48:34] ENGINE Bus STARTING 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: [10/Sep/2024:22:48:34] ENGINE Serving on http://172.21.15.67:8765 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: [10/Sep/2024:22:48:34] ENGINE Serving on https://172.21.15.67:7150 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: [10/Sep/2024:22:48:34] ENGINE Bus STARTED 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: [10/Sep/2024:22:48:34] ENGINE Error in HTTPServer.serve 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout: Traceback (most recent call last): 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:48:35.351 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._run(expiration_interval) 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: s = self.context.wrap_socket( 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: return self.sslsocket_class._create( 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: self.do_handshake() 2024-09-10T22:48:35.352 INFO:journalctl@ceph.mon.c.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:48:35.353 INFO:journalctl@ceph.mon.c.smithi145.stdout: self._sslobj.do_handshake() 2024-09-10T22:48:35.353 INFO:journalctl@ceph.mon.c.smithi145.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:48:35.353 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.353 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: mgrmap e17: x(active, since 2s) 2024-09-10T22:48:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: [10/Sep/2024:22:48:34] ENGINE Bus STARTING 2024-09-10T22:48:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: [10/Sep/2024:22:48:34] ENGINE Serving on http://172.21.15.67:8765 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: [10/Sep/2024:22:48:34] ENGINE Serving on https://172.21.15.67:7150 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: [10/Sep/2024:22:48:34] ENGINE Bus STARTED 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: [10/Sep/2024:22:48:34] ENGINE Error in HTTPServer.serve 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout: Traceback (most recent call last): 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:48:35.523 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: self._run(expiration_interval) 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: s = self.context.wrap_socket( 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: return self.sslsocket_class._create( 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: self.do_handshake() 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:48:35.524 INFO:journalctl@ceph.mon.b.smithi092.stdout: self._sslobj.do_handshake() 2024-09-10T22:48:35.525 INFO:journalctl@ceph.mon.b.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:48:35.525 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:35.525 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.252 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:36.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.858+0000 7f443a000640 1 -- 172.21.15.67:0/863462577 <== mon.2 v2:172.21.15.92:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v12)=0 v12) ==== 109+0+0 (secure 0 0 0) 0x7f4428021e00 con 0x7f443c103b90 2024-09-10T22:48:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.863+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f442403f5b0 msgr2=0x7f4424041a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.863+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f442403f5b0 0x7f4424041a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4434006010 tx=0x7f44340316d0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.863+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f443c103b90 msgr2=0x7f443c19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.863+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f443c103b90 0x7f443c19b290 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800edf0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 shutdown_connections 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f442403f5b0 0x7f4424041a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f443c104550 0x7f443c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f443c103b90 0x7f443c19b290 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 --2- 172.21.15.67:0/863462577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f443c102a30 0x7f443c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.865+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 >> 172.21.15.67:0/863462577 conn(0x7f443c0fe320 msgr2=0x7f443c1094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:36.869 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:36.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.868+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 shutdown_connections 2024-09-10T22:48:36.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:36.868+0000 7f4443e00640 1 -- 172.21.15.67:0/863462577 wait complete. 2024-09-10T22:48:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:37.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:37 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:37 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:37 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:37 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:37 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:37 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:37 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:37 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:37 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 -- 172.21.15.67:0/1694502246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4103b90 msgr2=0x7fd2f4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 --2- 172.21.15.67:0/1694502246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4103b90 0x7fd2f4104010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009930 tx=0x7fd2e002a420 comp rx=0 tx=0).stop 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 -- 172.21.15.67:0/1694502246 shutdown_connections 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 --2- 172.21.15.67:0/1694502246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4104550 0x7fd2f410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 --2- 172.21.15.67:0/1694502246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4103b90 0x7fd2f4104010 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 --2- 172.21.15.67:0/1694502246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4102a30 0x7fd2f4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.243+0000 7fd2fac00640 1 -- 172.21.15.67:0/1694502246 >> 172.21.15.67:0/1694502246 conn(0x7fd2f40fe320 msgr2=0x7fd2f4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.244+0000 7fd2fac00640 1 -- 172.21.15.67:0/1694502246 shutdown_connections 2024-09-10T22:48:38.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.244+0000 7fd2fac00640 1 -- 172.21.15.67:0/1694502246 wait complete. 2024-09-10T22:48:38.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 Processor -- start 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 -- start start 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 0x7fd2f4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4104550 0x7fd2f4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f410e380 con 0x7fd2f4102a30 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f410e200 con 0x7fd2f4103b90 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.245+0000 7fd2fac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f410e500 con 0x7fd2f4104550 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53526/0 (socket says 172.21.15.67:53526) 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 -- 172.21.15.67:0/2180566761 learned_addr learned my addr 172.21.15.67:0/2180566761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:38.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f8c00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4104550 0x7fd2f4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:38.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 -- 172.21.15.67:0/2180566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4104550 msgr2=0x7fd2f4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:38.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4104550 0x7fd2f4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 -- 172.21.15.67:0/2180566761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 msgr2=0x7fd2f4075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3400640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 0x7fd2f4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 0x7fd2f4075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3e00640 1 -- 172.21.15.67:0/2180566761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f4076340 con 0x7fd2f4102a30 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.246+0000 7fd2f3400640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 0x7fd2f4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2f3e00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e400d950 tx=0x7fd2e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4017030 con 0x7fd2f4102a30 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1007+0+0 (secure 0 0 0) 0x7fd2e4004020 con 0x7fd2f4102a30 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f41a2c50 con 0x7fd2f4102a30 2024-09-10T22:48:38.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4005450 con 0x7fd2f4102a30 2024-09-10T22:48:38.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.247+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f41a3160 con 0x7fd2f4102a30 2024-09-10T22:48:38.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.249+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f4109e80 con 0x7fd2f4102a30 2024-09-10T22:48:38.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.249+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fd2e4009a30 con 0x7fd2f4102a30 2024-09-10T22:48:38.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.249+0000 7fd2f0c00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 0x7fd2d8041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:38.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.250+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd2e4053990 con 0x7fd2f4102a30 2024-09-10T22:48:38.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.250+0000 7fd2f3400640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 0x7fd2d8041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:38.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.251+0000 7fd2f3400640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 0x7fd2d8041a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00096c0 tx=0x7fd2e0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:38.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.258+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd2e40041d0 con 0x7fd2f4102a30 2024-09-10T22:48:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:38 smithi067 ceph-mon[30939]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.459+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fd2f402e5c0 con 0x7fd2f4102a30 2024-09-10T22:48:38.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.463+0000 7fd2f0c00640 1 -- 172.21.15.67:0/2180566761 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v13)=0 v13) ==== 117+0+0 (secure 0 0 0) 0x7fd2f402e5c0 con 0x7fd2f4102a30 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.465+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 msgr2=0x7fd2d8041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.465+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 0x7fd2d8041a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00096c0 tx=0x7fd2e0005fd0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 msgr2=0x7fd2f4076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e400d950 tx=0x7fd2e400de10 comp rx=0 tx=0).stop 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 shutdown_connections 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2d803f560 0x7fd2d8041a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2f4104550 0x7fd2f4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f4103b90 0x7fd2f4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 --2- 172.21.15.67:0/2180566761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f4102a30 0x7fd2f4076cd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:38.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 >> 172.21.15.67:0/2180566761 conn(0x7fd2f40fe320 msgr2=0x7fd2f4107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:38.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 shutdown_connections 2024-09-10T22:48:38.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:38.466+0000 7fd2fac00640 1 -- 172.21.15.67:0/2180566761 wait complete. 2024-09-10T22:48:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:38 smithi092 ceph-mon[34223]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:38 smithi145 ceph-mon[34397]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2180566761' entity='client.admin' 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.576 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:39.577 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2180566761' entity='client.admin' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:39.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:39.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:39.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2180566761' entity='client.admin' 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: Updating smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:48:39.846 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:39.847 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:40.740 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:40 smithi067 ceph-mon[30939]: Deploying daemon mgr.z on smithi145 2024-09-10T22:48:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:40 smithi092 ceph-mon[34223]: Deploying daemon mgr.z on smithi145 2024-09-10T22:48:40.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:40 smithi145 ceph-mon[34397]: Deploying daemon mgr.z on smithi145 2024-09-10T22:48:41.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 -- 172.21.15.67:0/3391792770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc10f220 msgr2=0x7f16cc111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 --2- 172.21.15.67:0/3391792770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc10f220 0x7f16cc111610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009ca0 tx=0x7f16c402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 -- 172.21.15.67:0/3391792770 shutdown_connections 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 --2- 172.21.15.67:0/3391792770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc10f220 0x7f16cc111610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 --2- 172.21.15.67:0/3391792770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc075de0 0x7f16cc10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 --2- 172.21.15.67:0/3391792770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc0754c0 0x7f16cc0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 -- 172.21.15.67:0/3391792770 >> 172.21.15.67:0/3391792770 conn(0x7f16cc0fe660 msgr2=0x7f16cc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 -- 172.21.15.67:0/3391792770 shutdown_connections 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.738+0000 7f16d0a00640 1 -- 172.21.15.67:0/3391792770 wait complete. 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.739+0000 7f16d0a00640 1 Processor -- start 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.739+0000 7f16d0a00640 1 -- start start 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 0x7f16cc19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc10f220 0x7f16cc1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc114ac0 con 0x7f16cc075de0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc114940 con 0x7f16cc0754c0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16d0a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc114c40 con 0x7f16cc10f220 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47872/0 (socket says 172.21.15.67:47872) 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16c2a00640 1 -- 172.21.15.67:0/2707981826 learned_addr learned my addr 172.21.15.67:0/2707981826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16cbe00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc10f220 0x7f16cc1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.740+0000 7f16cb400640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 0x7f16cc19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 -- 172.21.15.67:0/2707981826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc10f220 msgr2=0x7f16cc1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc10f220 0x7f16cc1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 -- 172.21.15.67:0/2707981826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 msgr2=0x7f16cc19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 0x7f16cc19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 -- 172.21.15.67:0/2707981826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc1a1070 con 0x7f16cc075de0 2024-09-10T22:48:41.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16cb400640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 0x7f16cc19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c2a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cd80 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) ==== 1007+0+0 (secure 0 0 0) 0x7f16b4010880 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4018710 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc1a1360 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.741+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a7a20 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.743+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc0758a0 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.743+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f16b4010430 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.743+0000 7f16c8c00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 0x7f16b0041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.743+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f16b4054330 con 0x7f16cc075de0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.743+0000 7f16cb400640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 0x7f16b0041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.744+0000 7f16cb400640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 0x7f16b0041ad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005d00 tx=0x7f16bc005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.748+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f16b400b2c0 con 0x7f16cc075de0 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.949+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f16cc02e5c0 con 0x7f16cc075de0 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.953+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f16b40188b0 con 0x7f16cc075de0 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.954+0000 7f16c8c00640 1 -- 172.21.15.67:0/2707981826 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(24 keys) ==== 1038+0+0 (secure 0 0 0) 0x7f16b4010c10 con 0x7f16cc075de0 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.955+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 msgr2=0x7f16b0041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.955+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 0x7f16b0041ad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005d00 tx=0x7f16bc005c70 comp rx=0 tx=0).stop 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.955+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 msgr2=0x7f16cc19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.955+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 shutdown_connections 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16b003f610 0x7f16b0041ad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc10f220 0x7f16cc1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16cc075de0 0x7f16cc19b500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 --2- 172.21.15.67:0/2707981826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0754c0 0x7f16cc19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 >> 172.21.15.67:0/2707981826 conn(0x7f16cc0fe660 msgr2=0x7f16cc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 shutdown_connections 2024-09-10T22:48:41.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:40.956+0000 7f16d0a00640 1 -- 172.21.15.67:0/2707981826 wait complete. 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.645+0000 7f3aa4e00640 1 -- 172.21.15.67:0/988974275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108610 msgr2=0x7f3aa0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.645+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/988974275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108610 0x7f3aa0108a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009900 tx=0x7f3a8c02a410 comp rx=0 tx=0).stop 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 -- 172.21.15.67:0/988974275 shutdown_connections 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/988974275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108f90 0x7f3aa010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/988974275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108610 0x7f3aa0108a50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/988974275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 -- 172.21.15.67:0/988974275 >> 172.21.15.67:0/988974275 conn(0x7f3aa00fe6d0 msgr2=0x7f3aa0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 -- 172.21.15.67:0/988974275 shutdown_connections 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.646+0000 7f3aa4e00640 1 -- 172.21.15.67:0/988974275 wait complete. 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 Processor -- start 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 -- start start 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 0x7f3aa019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 0x7f3aa01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112a10 con 0x7f3aa0108f90 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112890 con 0x7f3aa0102c00 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3aa4e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112b90 con 0x7f3aa0108610 2024-09-10T22:48:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3a9f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3a9fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 0x7f3aa01a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 0x7f3aa019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3a9f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa019b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:35402/0 (socket says 172.21.15.67:35402) 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.647+0000 7f3a9f400640 1 -- 172.21.15.67:0/2513291311 learned_addr learned my addr 172.21.15.67:0/2513291311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 -- 172.21.15.67:0/2513291311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 msgr2=0x7f3aa019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 -- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 msgr2=0x7f3aa01a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 0x7f3aa01a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 -- 172.21.15.67:0/2513291311 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa01a10e0 con 0x7f3aa0108610 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9fe00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 0x7f3aa01a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.648+0000 7f3a9ea00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 0x7f3aa019b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c02aa00 tx=0x7f3a8c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.649+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c04d070 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.649+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(24 keys) ==== 1038+0+0 (secure 0 0 0) 0x7f3a8c031870 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.649+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c035070 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.649+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa01a1370 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.649+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa0109660 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.650+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f3a8c047030 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.650+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa0103eb0 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.650+0000 7f3a97e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 0x7f3a84041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.651+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3a8c07c410 con 0x7f3aa0108610 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.651+0000 7f3a9f400640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 0x7f3a84041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.651+0000 7f3a9f400640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 0x7f3a84041ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ac90 tx=0x7f3a9000a380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.656+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3a8c045070 con 0x7f3aa0108610 2024-09-10T22:48:41.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.858+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f3aa002e5c0 con 0x7f3aa0108610 2024-09-10T22:48:41.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.863+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 7 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a8c047c00 con 0x7f3aa0108610 2024-09-10T22:48:41.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.863+0000 7f3a97e00640 1 -- 172.21.15.67:0/2513291311 <== mon.1 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v15)=0 v15) ==== 114+0+0 (secure 0 0 0) 0x7f3a8c052560 con 0x7f3aa0108610 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 msgr2=0x7f3a84041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 0x7f3a84041ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ac90 tx=0x7f3a9000a380 comp rx=0 tx=0).stop 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 msgr2=0x7f3aa019b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 0x7f3aa019b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c02aa00 tx=0x7f3a8c0038a0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 shutdown_connections 2024-09-10T22:48:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a8403f610 0x7f3a84041ad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3aa0108f90 0x7f3aa01a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3aa0108610 0x7f3aa019b570 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 --2- 172.21.15.67:0/2513291311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aa0102c00 0x7f3aa019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 >> 172.21.15.67:0/2513291311 conn(0x7f3aa00fe6d0 msgr2=0x7f3aa0105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 shutdown_connections 2024-09-10T22:48:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:41.867+0000 7f3aa4e00640 1 -- 172.21.15.67:0/2513291311 wait complete. 2024-09-10T22:48:41.880 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:41 smithi145 systemd[1]: Starting Ceph mgr.z for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:48:42.181 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:41 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2707981826' entity='client.admin' 2024-09-10T22:48:42.181 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:41 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:42.182 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:41 smithi145 podman[37016]: 2024-09-10 22:48:41.829286781 +0000 UTC m=+0.027256468 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:48:42.182 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:41 smithi145 podman[37016]: 2024-09-10 22:48:41.961341707 +0000 UTC m=+0.159311380 container create 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS) 2024-09-10T22:48:42.182 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:42 smithi145 podman[37016]: 2024-09-10 22:48:42.1809133 +0000 UTC m=+0.378882973 container init 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T22:48:42.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:41 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2707981826' entity='client.admin' 2024-09-10T22:48:42.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:41 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:41 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2707981826' entity='client.admin' 2024-09-10T22:48:42.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:41 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:42.630 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:42 smithi145 podman[37016]: 2024-09-10 22:48:42.187069144 +0000 UTC m=+0.385038817 container start 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:48:42.630 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:42 smithi145 ceph-mgr[37035]: -- 172.21.15.145:0/981627667 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a62ce90e0 con 0x560a62cc6800 2024-09-10T22:48:42.631 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:42 smithi145 bash[37016]: 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 2024-09-10T22:48:42.631 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 22:48:42 smithi145 systemd[1]: Started Ceph mgr.z for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:48:42.688 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.041 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.042 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 -- 172.21.15.67:0/2655403465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb241087d0 msgr2=0x7fdb24108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:43.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 --2- 172.21.15.67:0/2655403465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb241087d0 0x7fdb24108c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009ca0 tx=0x7fdb0c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 -- 172.21.15.67:0/2655403465 shutdown_connections 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 --2- 172.21.15.67:0/2655403465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 0x7fdb2410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 --2- 172.21.15.67:0/2655403465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb241087d0 0x7fdb24108c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 --2- 172.21.15.67:0/2655403465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb24102dc0 0x7fdb241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 -- 172.21.15.67:0/2655403465 >> 172.21.15.67:0/2655403465 conn(0x7fdb240fe830 msgr2=0x7fdb24100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 -- 172.21.15.67:0/2655403465 shutdown_connections 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.004+0000 7fdb29a00640 1 -- 172.21.15.67:0/2655403465 wait complete. 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.005+0000 7fdb29a00640 1 Processor -- start 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- start start 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 0x7fdb2419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 0x7fdb241a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24112d10 con 0x7fdb24102dc0 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24112b90 con 0x7fdb241087d0 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24112e90 con 0x7fdb24109150 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:43.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47902/0 (socket says 172.21.15.67:47902) 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 -- 172.21.15.67:0/848369452 learned_addr learned my addr 172.21.15.67:0/848369452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb23400640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 0x7fdb241a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb1a000640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 0x7fdb2419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 -- 172.21.15.67:0/848369452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 msgr2=0x7fdb241a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 0x7fdb241a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 -- 172.21.15.67:0/848369452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 msgr2=0x7fdb2419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 0x7fdb2419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 -- 172.21.15.67:0/848369452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241a1480 con 0x7fdb24102dc0 2024-09-10T22:48:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb1a000640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 0x7fdb2419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb22a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400d950 tx=0x7fdb1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14017030 con 0x7fdb24102dc0 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb14004020 con 0x7fdb24102dc0 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241a1770 con 0x7fdb24102dc0 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140055d0 con 0x7fdb24102dc0 2024-09-10T22:48:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.007+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241a7e30 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.008+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb241031a0 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.008+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fdb14009a30 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.010+0000 7fdb1be00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 0x7fdb08041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.010+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdb14053810 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.010+0000 7fdb1a000640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 0x7fdb08041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.011+0000 7fdb1a000640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 0x7fdb08041a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb24107f00 tx=0x7fdb0c009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.014+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb1401ad70 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.221+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fdb2402e5c0 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.225+0000 7fdb1be00640 1 -- 172.21.15.67:0/848369452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fdb140041d0 con 0x7fdb24102dc0 2024-09-10T22:48:43.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 msgr2=0x7fdb08041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 0x7fdb08041a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdb24107f00 tx=0x7fdb0c009360 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 msgr2=0x7fdb2419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400d950 tx=0x7fdb1400de10 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 shutdown_connections 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdb0803f560 0x7fdb08041a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb24109150 0x7fdb241a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb241087d0 0x7fdb2419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 --2- 172.21.15.67:0/848369452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb24102dc0 0x7fdb2419b3d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.227+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 >> 172.21.15.67:0/848369452 conn(0x7fdb240fe830 msgr2=0x7fdb24104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.228+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 shutdown_connections 2024-09-10T22:48:43.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:42.228+0000 7fdb29a00640 1 -- 172.21.15.67:0/848369452 wait complete. 2024-09-10T22:48:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/848369452' entity='client.admin' 2024-09-10T22:48:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:43.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:43 smithi092 ceph-mon[34223]: Deploying daemon mgr.y on smithi092 2024-09-10T22:48:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/848369452' entity='client.admin' 2024-09-10T22:48:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:43 smithi067 ceph-mon[30939]: Deploying daemon mgr.y on smithi092 2024-09-10T22:48:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/848369452' entity='client.admin' 2024-09-10T22:48:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:48:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:43 smithi145 ceph-mon[34397]: Deploying daemon mgr.y on smithi092 2024-09-10T22:48:43.776 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.776 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi067') 2024-09-10T22:48:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.864 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.865 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.904 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.905 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:43.915 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.032 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.075 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.116 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.120 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.121 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.139 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.169 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:44.898 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:44 smithi092 systemd[1]: Starting Ceph mgr.y for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:48:45.160 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:44 smithi092 podman[36533]: 2024-09-10 22:48:44.800993821 +0000 UTC m=+0.027066198 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:48:45.160 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:44 smithi092 podman[36533]: 2024-09-10 22:48:44.909998302 +0000 UTC m=+0.136070649 container create a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T22:48:45.160 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:45 smithi092 podman[36533]: 2024-09-10 22:48:45.112471289 +0000 UTC m=+0.338543658 container init a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T22:48:45.160 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:45 smithi092 podman[36533]: 2024-09-10 22:48:45.119221629 +0000 UTC m=+0.345293983 container start a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:48:45.161 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:45 smithi092 ceph-mgr[36552]: -- 172.21.15.92:0/4053060400 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583f5b5d0e0 con 0x5583f5b3a800 2024-09-10T22:48:45.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:45 smithi092 bash[36533]: a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b 2024-09-10T22:48:45.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 22:48:45 smithi092 systemd[1]: Started Ceph mgr.y for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:48:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 -- 172.21.15.67:0/2935337906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10b340 msgr2=0x7fcd5c111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 --2- 172.21.15.67:0/2935337906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10b340 0x7fcd5c111b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ca0 tx=0x7fcd5402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 -- 172.21.15.67:0/2935337906 shutdown_connections 2024-09-10T22:48:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 --2- 172.21.15.67:0/2935337906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10b340 0x7fcd5c111b40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 --2- 172.21.15.67:0/2935337906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 --2- 172.21.15.67:0/2935337906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 -- 172.21.15.67:0/2935337906 >> 172.21.15.67:0/2935337906 conn(0x7fcd5c100a20 msgr2=0x7fcd5c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 -- 172.21.15.67:0/2935337906 shutdown_connections 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.781+0000 7fcd60e00640 1 -- 172.21.15.67:0/2935337906 wait complete. 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.782+0000 7fcd60e00640 1 Processor -- start 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.782+0000 7fcd60e00640 1 -- start start 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c10b340 0x7fcd5c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114e70 con 0x7fcd5c10a9c0 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114cf0 con 0x7fcd5c104fb0 2024-09-10T22:48:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd60e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114ff0 con 0x7fcd5c10b340 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47920/0 (socket says 172.21.15.67:47920) 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59400640 1 -- 172.21.15.67:0/316024323 learned_addr learned my addr 172.21.15.67:0/316024323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd5a800640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c10b340 0x7fcd5c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 -- 172.21.15.67:0/316024323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c10b340 msgr2=0x7fcd5c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c10b340 0x7fcd5c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 -- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 msgr2=0x7fcd5c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59e00640 1 -- 172.21.15.67:0/316024323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c1a1430 con 0x7fcd5c104fb0 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.783+0000 7fcd59400640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd59e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c19b380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d720 tx=0x7fcd4c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c017070 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd4c00de30 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0047a0 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c1a1720 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.784+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a7de0 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.785+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c105390 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.785+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fcd4c0099d0 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.786+0000 7fcd52a00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 0x7fcd40041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.786+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcd4c053b40 con 0x7fcd5c104fb0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.786+0000 7fcd59400640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 0x7fcd40041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.787+0000 7fcd59400640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 0x7fcd40041a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c107470 tx=0x7fcd48005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.791+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcd4c020020 con 0x7fcd5c104fb0 2024-09-10T22:48:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.989+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fcd5c02e5c0 con 0x7fcd5c104fb0 2024-09-10T22:48:45.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.995+0000 7fcd52a00640 1 -- 172.21.15.67:0/316024323 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) ==== 106+0+0 (secure 0 0 0) 0x7fcd4c018270 con 0x7fcd5c104fb0 2024-09-10T22:48:45.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 msgr2=0x7fcd40041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 0x7fcd40041a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c107470 tx=0x7fcd48005f90 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 msgr2=0x7fcd5c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c19b380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d720 tx=0x7fcd4c00dbe0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 shutdown_connections 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcd4003f560 0x7fcd40041a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd5c10b340 0x7fcd5c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd5c10a9c0 0x7fcd5c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 --2- 172.21.15.67:0/316024323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c104fb0 0x7fcd5c19b380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 >> 172.21.15.67:0/316024323 conn(0x7fcd5c100a20 msgr2=0x7fcd5c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 shutdown_connections 2024-09-10T22:48:45.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:45.997+0000 7fcd60e00640 1 -- 172.21.15.67:0/316024323 wait complete. 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:46 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:46 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 -- 172.21.15.67:0/870702581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f81087b0 msgr2=0x7f79f8108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 --2- 172.21.15.67:0/870702581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f81087b0 0x7f79f8108bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 -- 172.21.15.67:0/870702581 shutdown_connections 2024-09-10T22:48:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 --2- 172.21.15.67:0/870702581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f8109130 0x7f79f810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 --2- 172.21.15.67:0/870702581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f81087b0 0x7f79f8108bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 --2- 172.21.15.67:0/870702581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 -- 172.21.15.67:0/870702581 >> 172.21.15.67:0/870702581 conn(0x7f79f80fe810 msgr2=0x7f79f8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 -- 172.21.15.67:0/870702581 shutdown_connections 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.721+0000 7f79ff200640 1 -- 172.21.15.67:0/870702581 wait complete. 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 Processor -- start 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 -- start start 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f81087b0 0x7f79f81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 0x7f79f819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8112cf0 con 0x7f79f8109130 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8112b70 con 0x7f79f81087b0 2024-09-10T22:48:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.723+0000 7f79ff200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8112e70 con 0x7f79f8102da0 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 0x7f79f819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:58932/0 (socket says 172.21.15.67:58932) 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79f7e00640 1 -- 172.21.15.67:0/190511417 learned_addr learned my addr 172.21.15.67:0/190511417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79ef400640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f81087b0 0x7f79f81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 -- 172.21.15.67:0/190511417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 msgr2=0x7f79f81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 -- 172.21.15.67:0/190511417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f81087b0 msgr2=0x7f79f81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f81087b0 0x7f79f81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 -- 172.21.15.67:0/190511417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f819f250 con 0x7f79f8109130 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79f7e00640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.724+0000 7f79fd200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 0x7f79f819ea80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79f000da10 tx=0x7f79f000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.725+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f000bd70 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.725+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79f0003e00 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.725+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f00106c0 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.725+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f819f540 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.725+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f81a5c00 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.726+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f79f0024060 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.726+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f8103180 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.726+0000 7f79f5600640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 0x7f79dc041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.727+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f79f0053e60 con 0x7f79f8109130 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.727+0000 7f79f7e00640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 0x7f79dc041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.727+0000 7f79f7e00640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 0x7f79dc041a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011500 tx=0x7f79e8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.731+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f79f0004270 con 0x7f79f8109130 2024-09-10T22:48:46.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:46 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:46.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.934+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f79f802e5c0 con 0x7f79f8109130 2024-09-10T22:48:46.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.939+0000 7f79f5600640 1 -- 172.21.15.67:0/190511417 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7f79f001c400 con 0x7f79f8109130 2024-09-10T22:48:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 msgr2=0x7f79dc041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 0x7f79dc041a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011500 tx=0x7f79e8005f90 comp rx=0 tx=0).stop 2024-09-10T22:48:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 msgr2=0x7f79f819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 0x7f79f819ea80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79f000da10 tx=0x7f79f000ded0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 shutdown_connections 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f79dc03f560 0x7f79dc041a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79f8109130 0x7f79f819ea80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79f81087b0 0x7f79f81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 --2- 172.21.15.67:0/190511417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79f8102da0 0x7f79f81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 >> 172.21.15.67:0/190511417 conn(0x7f79f80fe810 msgr2=0x7f79f8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 shutdown_connections 2024-09-10T22:48:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:46.942+0000 7f79ff200640 1 -- 172.21.15.67:0/190511417 wait complete. 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 -- 172.21.15.67:0/1276278088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10a9d0 msgr2=0x7f3a4c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 --2- 172.21.15.67:0/1276278088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c10ae10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009900 tx=0x7f3a3402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 -- 172.21.15.67:0/1276278088 shutdown_connections 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 --2- 172.21.15.67:0/1276278088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10b350 0x7f3a4c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 --2- 172.21.15.67:0/1276278088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c10ae10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 --2- 172.21.15.67:0/1276278088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.005+0000 7f3a51c00640 1 -- 172.21.15.67:0/1276278088 >> 172.21.15.67:0/1276278088 conn(0x7f3a4c1009f0 msgr2=0x7f3a4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.006+0000 7f3a51c00640 1 -- 172.21.15.67:0/1276278088 shutdown_connections 2024-09-10T22:48:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.006+0000 7f3a51c00640 1 -- 172.21.15.67:0/1276278088 wait complete. 2024-09-10T22:48:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 Processor -- start 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 -- start start 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10b350 0x7f3a4c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c114e70 con 0x7f3a4c10b350 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c114cf0 con 0x7f3a4c10a9d0 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.007+0000 7f3a51c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c114ff0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a42000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:58962/0 (socket says 172.21.15.67:58962) 2024-09-10T22:48:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 -- 172.21.15.67:0/2920726031 learned_addr learned my addr 172.21.15.67:0/2920726031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4b400640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10b350 0x7f3a4c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 -- 172.21.15.67:0/2920726031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 msgr2=0x7f3a4c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 -- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10b350 msgr2=0x7f3a4c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10b350 0x7f3a4c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a4aa00640 1 -- 172.21.15.67:0/2920726031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1a1470 con 0x7f3a4c104fc0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.008+0000 7f3a42000640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a4aa00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00eb90 tx=0x7f3a3c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004d20 con 0x7f3a4c104fc0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a3c00ee90 con 0x7f3a4c104fc0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c010690 con 0x7f3a4c104fc0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1a1760 con 0x7f3a4c104fc0 2024-09-10T22:48:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.009+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1a7e20 con 0x7f3a4c104fc0 2024-09-10T22:48:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.010+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f3a3c0108f0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.011+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c1053a0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.011+0000 7f3a43e00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 0x7f3a30041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.011+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3a3c0545f0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.012+0000 7f3a42000640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 0x7f3a30041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.012+0000 7f3a42000640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 0x7f3a30041a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a34006010 tx=0x7f3a34000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.016+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3a3c00f3c0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.221+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f3a4c02e5c0 con 0x7f3a4c104fc0 2024-09-10T22:48:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.226+0000 7f3a43e00640 1 -- 172.21.15.67:0/2920726031 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v19)=0 v19) ==== 129+0+0 (secure 0 0 0) 0x7f3a3c020e10 con 0x7f3a4c104fc0 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 msgr2=0x7f3a30041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 0x7f3a30041a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a34006010 tx=0x7f3a34000bc0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 msgr2=0x7f3a4c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00eb90 tx=0x7f3a3c00c6a0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 shutdown_connections 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3a3003f560 0x7f3a30041a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a4c10b350 0x7f3a4c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c10a9d0 0x7f3a4c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 --2- 172.21.15.67:0/2920726031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c104fc0 0x7f3a4c19b3c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 >> 172.21.15.67:0/2920726031 conn(0x7f3a4c1009f0 msgr2=0x7f3a4c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:47.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.229+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 shutdown_connections 2024-09-10T22:48:47.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.230+0000 7f3a51c00640 1 -- 172.21.15.67:0/2920726031 wait complete. 2024-09-10T22:48:47.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.418+0000 7fe709a00640 1 -- 172.21.15.67:0/2665181763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe704104fc0 msgr2=0x7fe7041053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.418+0000 7fe709a00640 1 --2- 172.21.15.67:0/2665181763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe704104fc0 0x7fe7041053a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009900 tx=0x7fe6f402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 -- 172.21.15.67:0/2665181763 shutdown_connections 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 --2- 172.21.15.67:0/2665181763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410b350 0x7fe704111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 --2- 172.21.15.67:0/2665181763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70410a9d0 0x7fe70410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 --2- 172.21.15.67:0/2665181763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe704104fc0 0x7fe7041053a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 -- 172.21.15.67:0/2665181763 >> 172.21.15.67:0/2665181763 conn(0x7fe7041009f0 msgr2=0x7fe704102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 -- 172.21.15.67:0/2665181763 shutdown_connections 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 -- 172.21.15.67:0/2665181763 wait complete. 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.419+0000 7fe709a00640 1 Processor -- start 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 -- start start 2024-09-10T22:48:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 0x7fe70419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 0x7fe7041a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe704114e80 con 0x7fe70410b350 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe704114d00 con 0x7fe704104fc0 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe709a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe704115000 con 0x7fe70410a9d0 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe703400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 0x7fe7041a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe702a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe702000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 0x7fe70419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe702a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:35508/0 (socket says 172.21.15.67:35508) 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.420+0000 7fe702a00640 1 -- 172.21.15.67:0/491196174 learned_addr learned my addr 172.21.15.67:0/491196174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 -- 172.21.15.67:0/491196174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 msgr2=0x7fe70419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 0x7fe70419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 -- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 msgr2=0x7fe7041a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 0x7fe7041a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 -- 172.21.15.67:0/491196174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7041a1500 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702000640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 0x7fe70419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe703400640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 0x7fe7041a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe702a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6f402a930 tx=0x7fe6f4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f403f070 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f403baf0 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.421+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4035030 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.422+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7041a1790 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.422+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7041a7eb0 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.423+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7041053a0 con 0x7fe704104fc0 2024-09-10T22:48:47.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.423+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fe6f403b660 con 0x7fe704104fc0 2024-09-10T22:48:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.423+0000 7fe6fb400640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 0x7fe6e8041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.423+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe6f4074940 con 0x7fe704104fc0 2024-09-10T22:48:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.423+0000 7fe702000640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 0x7fe6e8041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.424+0000 7fe702000640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 0x7fe6e8041a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe704103300 tx=0x7fe6f000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.428+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6f4031400 con 0x7fe704104fc0 2024-09-10T22:48:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.645+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fe70402e5c0 con 0x7fe704104fc0 2024-09-10T22:48:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.652+0000 7fe6fb400640 1 -- 172.21.15.67:0/491196174 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fe6f4039d00 con 0x7fe704104fc0 2024-09-10T22:48:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 msgr2=0x7fe6e8041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 0x7fe6e8041a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe704103300 tx=0x7fe6f000b410 comp rx=0 tx=0).stop 2024-09-10T22:48:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 msgr2=0x7fe70419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6f402a930 tx=0x7fe6f4003b90 comp rx=0 tx=0).stop 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 shutdown_connections 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6e803f560 0x7fe6e8041a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70410b350 0x7fe7041a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe70410a9d0 0x7fe70419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 --2- 172.21.15.67:0/491196174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe704104fc0 0x7fe70419b450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 >> 172.21.15.67:0/491196174 conn(0x7fe7041009f0 msgr2=0x7fe704101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 shutdown_connections 2024-09-10T22:48:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:47.654+0000 7fe709a00640 1 -- 172.21.15.67:0/491196174 wait complete. 2024-09-10T22:48:47.975 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/190511417' entity='client.admin' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:47.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:47.977 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:47.977 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:47.977 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:47 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/190511417' entity='client.admin' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:47 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/190511417' entity='client.admin' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:48.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:48.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:48:48.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:47 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 -- 172.21.15.67:0/4097500586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4270104fc0 msgr2=0x7f42701053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 --2- 172.21.15.67:0/4097500586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4270104fc0 0x7f42701053a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4260009ca0 tx=0x7f426002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 -- 172.21.15.67:0/4097500586 shutdown_connections 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 --2- 172.21.15.67:0/4097500586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010b350 0x7f4270111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 --2- 172.21.15.67:0/4097500586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f427010a9d0 0x7f427010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 --2- 172.21.15.67:0/4097500586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4270104fc0 0x7f42701053a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 -- 172.21.15.67:0/4097500586 >> 172.21.15.67:0/4097500586 conn(0x7f42701009f0 msgr2=0x7f4270102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 -- 172.21.15.67:0/4097500586 shutdown_connections 2024-09-10T22:48:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.828+0000 7f4276a00640 1 -- 172.21.15.67:0/4097500586 wait complete. 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.829+0000 7f4276a00640 1 Processor -- start 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.829+0000 7f4276a00640 1 -- start start 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010a9d0 0x7f427019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427010b350 0x7f42701a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4270114ee0 con 0x7f427010b350 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4270114d60 con 0x7f4270104fc0 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4276a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4270115060 con 0x7f427010a9d0 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52558/0 (socket says 172.21.15.67:52558) 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 -- 172.21.15.67:0/2090980588 learned_addr learned my addr 172.21.15.67:0/2090980588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f4274a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427010b350 0x7f42701a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426f400640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010a9d0 0x7f427019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 -- 172.21.15.67:0/2090980588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010a9d0 msgr2=0x7f427019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010a9d0 0x7f427019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 -- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427010b350 msgr2=0x7f42701a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427010b350 0x7f42701a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.830+0000 7f426fe00640 1 -- 172.21.15.67:0/2090980588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42701a14a0 con 0x7f4270104fc0 2024-09-10T22:48:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f426fe00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4260031040 tx=0x7f4260032bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260004040 con 0x7f4270104fc0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4260004670 con 0x7f4270104fc0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701a1730 con 0x7f4270104fc0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260030d50 con 0x7f4270104fc0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.831+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a7e50 con 0x7f4270104fc0 2024-09-10T22:48:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.832+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42701053a0 con 0x7f4270104fc0 2024-09-10T22:48:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.833+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f42600041e0 con 0x7f4270104fc0 2024-09-10T22:48:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.834+0000 7f426cc00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 0x7f425407c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.834+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f42600bd1b0 con 0x7f4270104fc0 2024-09-10T22:48:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.834+0000 7f426f400640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 0x7f425407c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.835+0000 7f426f400640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 0x7f425407c8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f427010a100 tx=0x7f425c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:48.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:48.838+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4260004820 con 0x7f4270104fc0 2024-09-10T22:48:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.033+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f427002e5c0 con 0x7f4270104fc0 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: Reconfiguring daemon mgr.x on smithi067 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: Standby manager daemon z started 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:48:49.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:48 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.040+0000 7f426cc00640 1 -- 172.21.15.67:0/2090980588 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v21)=0 v21) ==== 116+0+0 (secure 0 0 0) 0x7f4260085830 con 0x7f4270104fc0 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 msgr2=0x7f425407c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 0x7f425407c8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f427010a100 tx=0x7f425c00b390 comp rx=0 tx=0).stop 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 msgr2=0x7f427019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4260031040 tx=0x7f4260032bc0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 shutdown_connections 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f425407a400 0x7f425407c8c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f427010b350 0x7f42701a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427010a9d0 0x7f427019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 --2- 172.21.15.67:0/2090980588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4270104fc0 0x7f427019b3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 >> 172.21.15.67:0/2090980588 conn(0x7f42701009f0 msgr2=0x7f4270102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 shutdown_connections 2024-09-10T22:48:49.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.043+0000 7f4276a00640 1 -- 172.21.15.67:0/2090980588 wait complete. 2024-09-10T22:48:49.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: Reconfiguring daemon mgr.x on smithi067 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: Standby manager daemon z started 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:48:49.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:48 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:48:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: Reconfiguring daemon mgr.x on smithi067 2024-09-10T22:48:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: Standby manager daemon z started 2024-09-10T22:48:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:48:49.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:49.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:48:49.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:48 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.145:0/2336211270' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:49.776 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi067') 2024-09-10T22:48:49.986 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:49 smithi067 ceph-mon[30939]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T22:48:49.986 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:48:49.986 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:49 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:49.987 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:49 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:49 smithi092 ceph-mon[34223]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T22:48:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:48:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:49 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:49 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:49 smithi145 ceph-mon[34397]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T22:48:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:48:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:49 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:49 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 -- 172.21.15.67:0/2362520295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 msgr2=0x7f973c108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 --2- 172.21.15.67:0/2362520295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c108a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9724009900 tx=0x7f972402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 -- 172.21.15.67:0/2362520295 shutdown_connections 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 --2- 172.21.15.67:0/2362520295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 --2- 172.21.15.67:0/2362520295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c108a60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 --2- 172.21.15.67:0/2362520295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 -- 172.21.15.67:0/2362520295 >> 172.21.15.67:0/2362520295 conn(0x7f973c0fe660 msgr2=0x7f973c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 -- 172.21.15.67:0/2362520295 shutdown_connections 2024-09-10T22:48:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.632+0000 7f973b200640 1 -- 172.21.15.67:0/2362520295 wait complete. 2024-09-10T22:48:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.633+0000 7f973b200640 1 Processor -- start 2024-09-10T22:48:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.633+0000 7f973b200640 1 -- start start 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f973c112af0 con 0x7f973c108620 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f973c112970 con 0x7f973c102c10 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f973c112c70 con 0x7f973c108fa0 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9731400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9731400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36670/0 (socket says 172.21.15.67:36670) 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9731400640 1 -- 172.21.15.67:0/300060868 learned_addr learned my addr 172.21.15.67:0/300060868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9739e00640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f973a800640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9739e00640 1 -- 172.21.15.67:0/300060868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 msgr2=0x7f973c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.634+0000 7f9739e00640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9739e00640 1 -- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 msgr2=0x7f973c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9739e00640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9739e00640 1 -- 172.21.15.67:0/300060868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c1a7300 con 0x7f973c102c10 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9731400640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f973a800640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9739e00640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c0754a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f972c005fd0 tx=0x7f972c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c009c30 con 0x7f973c102c10 2024-09-10T22:48:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c1a75f0 con 0x7f973c102c10 2024-09-10T22:48:50.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f972c0044c0 con 0x7f973c102c10 2024-09-10T22:48:50.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c003d70 con 0x7f973c102c10 2024-09-10T22:48:50.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.635+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c1a7b80 con 0x7f973c102c10 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.636+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c102ff0 con 0x7f973c102c10 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.638+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f972c004000 con 0x7f973c102c10 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.639+0000 7f9733400640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 0x7f972007c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.639+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f972c09be30 con 0x7f973c102c10 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.639+0000 7f9731400640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 0x7f972007c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.639+0000 7f9731400640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 0x7f972007c960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f97240096c0 tx=0x7f9724005e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:50.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.647+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f972c017020 con 0x7f973c102c10 2024-09-10T22:48:50.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.851+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f973c0767a0 con 0x7f973c102c10 2024-09-10T22:48:50.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.858+0000 7f9733400640 1 -- 172.21.15.67:0/300060868 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f972c064320 con 0x7f973c102c10 2024-09-10T22:48:50.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 msgr2=0x7f972007c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:50.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 0x7f972007c960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f97240096c0 tx=0x7f9724005e60 comp rx=0 tx=0).stop 2024-09-10T22:48:50.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 msgr2=0x7f973c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:50.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c0754a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f972c005fd0 tx=0x7f972c00dde0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 shutdown_connections 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f972007a4a0 0x7f972007c960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f973c108fa0 0x7f973c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973c108620 0x7f973c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 --2- 172.21.15.67:0/300060868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f973c102c10 0x7f973c0754a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 >> 172.21.15.67:0/300060868 conn(0x7f973c0fe660 msgr2=0x7f973c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 shutdown_connections 2024-09-10T22:48:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.860+0000 7f973b200640 1 -- 172.21.15.67:0/300060868 wait complete. 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 -- 172.21.15.67:0/25500746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d4075e00 msgr2=0x7f75d410ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 --2- 172.21.15.67:0/25500746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d4075e00 0x7f75d410ecc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009860 tx=0x7f75c002a340 comp rx=0 tx=0).stop 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 -- 172.21.15.67:0/25500746 shutdown_connections 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 --2- 172.21.15.67:0/25500746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 --2- 172.21.15.67:0/25500746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d4075e00 0x7f75d410ecc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 --2- 172.21.15.67:0/25500746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d40754e0 0x7f75d40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.394+0000 7f75dae00640 1 -- 172.21.15.67:0/25500746 >> 172.21.15.67:0/25500746 conn(0x7f75d40fe660 msgr2=0x7f75d4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.395+0000 7f75dae00640 1 -- 172.21.15.67:0/25500746 shutdown_connections 2024-09-10T22:48:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.395+0000 7f75dae00640 1 -- 172.21.15.67:0/25500746 wait complete. 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.395+0000 7f75dae00640 1 Processor -- start 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 -- start start 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 0x7f75d419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114c50 con 0x7f75d4075e00 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114ad0 con 0x7f75d40754e0 2024-09-10T22:48:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75dae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114dd0 con 0x7f75d410f200 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75d9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 0x7f75d419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75d9a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75d9a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52662/0 (socket says 172.21.15.67:52662) 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.396+0000 7f75d9a00640 1 -- 172.21.15.67:0/3381587780 learned_addr learned my addr 172.21.15.67:0/3381587780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75da400640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 -- 172.21.15.67:0/3381587780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 msgr2=0x7f75d41a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 -- 172.21.15.67:0/3381587780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 msgr2=0x7f75d419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 -- 172.21.15.67:0/3381587780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41a1230 con 0x7f75d4075e00 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75da400640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9a00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d9000640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 0x7f75d419b6c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75c002a860 tx=0x7f75c002aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0037040 con 0x7f75d4075e00 2024-09-10T22:48:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.397+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41a14c0 con 0x7f75d4075e00 2024-09-10T22:48:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.398+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c0030c30 con 0x7f75d4075e00 2024-09-10T22:48:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.398+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00035f0 con 0x7f75d4075e00 2024-09-10T22:48:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.398+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a7be0 con 0x7f75d4075e00 2024-09-10T22:48:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.399+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d40758c0 con 0x7f75d4075e00 2024-09-10T22:48:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.400+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f75c0004050 con 0x7f75d4075e00 2024-09-10T22:48:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.401+0000 7f75d2000640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 0x7f75bc07c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.401+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f75c00bb2d0 con 0x7f75d4075e00 2024-09-10T22:48:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.401+0000 7f75d9a00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 0x7f75bc07c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.402+0000 7f75d9a00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 0x7f75bc07c960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75c80058c0 tx=0x7f75c8009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: Standby manager daemon y started 2024-09-10T22:48:51.403 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:48:51.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:51.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:48:51.404 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:51 smithi067 ceph-mon[30939]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:51.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.406+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f75c004a060 con 0x7f75d4075e00 2024-09-10T22:48:51.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: Standby manager daemon y started 2024-09-10T22:48:51.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:48:51.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:51.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:48:51.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:51 smithi092 ceph-mon[34223]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.623+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f75d41a7ee0 con 0x7f75d4075e00 2024-09-10T22:48:51.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.627+0000 7f75d2000640 1 -- 172.21.15.67:0/3381587780 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v26)=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f75c0004370 con 0x7f75d4075e00 2024-09-10T22:48:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: Standby manager daemon y started 2024-09-10T22:48:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:48:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:48:51.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:48:51.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:51 smithi145 ceph-mon[34397]: from='mgr.? 172.21.15.92:0/228216760' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:48:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 msgr2=0x7f75bc07c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 0x7f75bc07c960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75c80058c0 tx=0x7f75c8009260 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 msgr2=0x7f75d419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 0x7f75d419b6c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75c002a860 tx=0x7f75c002aef0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 shutdown_connections 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.629+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75bc07a4a0 0x7f75bc07c960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d410f200 0x7f75d41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d4075e00 0x7f75d419b6c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 --2- 172.21.15.67:0/3381587780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d40754e0 0x7f75d419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 >> 172.21.15.67:0/3381587780 conn(0x7f75d40fe660 msgr2=0x7f75d40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 shutdown_connections 2024-09-10T22:48:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.630+0000 7f75dae00640 1 -- 172.21.15.67:0/3381587780 wait complete. 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.362+0000 7f8b91600640 1 -- 172.21.15.67:0/2484987123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 msgr2=0x7f8b8c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.362+0000 7f8b91600640 1 --2- 172.21.15.67:0/2484987123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c10ee10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009900 tx=0x7f8b7402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 -- 172.21.15.67:0/2484987123 shutdown_connections 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 --2- 172.21.15.67:0/2484987123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c10f3e0 0x7f8b8c1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 --2- 172.21.15.67:0/2484987123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c10ee10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 --2- 172.21.15.67:0/2484987123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c075540 0x7f8b8c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 -- 172.21.15.67:0/2484987123 >> 172.21.15.67:0/2484987123 conn(0x7f8b8c0fe830 msgr2=0x7f8b8c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 -- 172.21.15.67:0/2484987123 shutdown_connections 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.363+0000 7f8b91600640 1 -- 172.21.15.67:0/2484987123 wait complete. 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.364+0000 7f8b91600640 1 Processor -- start 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.364+0000 7f8b91600640 1 -- start start 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 0x7f8b8c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c10f3e0 0x7f8b8c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c114ea0 con 0x7f8b8c075f40 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c114d20 con 0x7f8b8c10f3e0 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b91600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c115020 con 0x7f8b8c075540 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36642/0 (socket says 172.21.15.67:36642) 2024-09-10T22:48:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b83e00640 1 -- 172.21.15.67:0/3131262439 learned_addr learned my addr 172.21.15.67:0/3131262439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b8aa00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 0x7f8b8c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b8b400640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c10f3e0 0x7f8b8c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.365+0000 7f8b8aa00640 1 -- 172.21.15.67:0/3131262439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c10f3e0 msgr2=0x7f8b8c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b8aa00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c10f3e0 0x7f8b8c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b8aa00640 1 -- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 msgr2=0x7f8b8c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b8aa00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b8aa00640 1 -- 172.21.15.67:0/3131262439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c1a14a0 con 0x7f8b8c075540 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b83e00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b8aa00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 0x7f8b8c19b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00e9f0 tx=0x7f8b7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.366+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00cd80 con 0x7f8b8c075540 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.367+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b7c0035f0 con 0x7f8b8c075540 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.367+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c010670 con 0x7f8b8c075540 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.367+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c1a1790 con 0x7f8b8c075540 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.367+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c1a7e50 con 0x7f8b8c075540 2024-09-10T22:48:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.368+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c075920 con 0x7f8b8c075540 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.369+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f8b7c004000 con 0x7f8b8c075540 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.370+0000 7f8b83400640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 0x7f8b7007c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.370+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8b7c09c140 con 0x7f8b8c075540 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.371+0000 7f8b83e00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 0x7f8b7007c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.371+0000 7f8b83e00640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 0x7f8b7007c910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b740096c0 tx=0x7f8b74005e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.375+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8b7c064730 con 0x7f8b8c075540 2024-09-10T22:48:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.584+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f8b8c1a8150 con 0x7f8b8c075540 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.590+0000 7f8b83400640 1 -- 172.21.15.67:0/3131262439 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v22)=0 v22) ==== 125+0+0 (secure 0 0 0) 0x7f8b7c069160 con 0x7f8b8c075540 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 msgr2=0x7f8b7007c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 0x7f8b7007c910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b740096c0 tx=0x7f8b74005e60 comp rx=0 tx=0).stop 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 msgr2=0x7f8b8c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 0x7f8b8c19b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00e9f0 tx=0x7f8b7c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 shutdown_connections 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8b7007a450 0x7f8b7007c910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b8c10f3e0 0x7f8b8c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8c075f40 0x7f8b8c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 --2- 172.21.15.67:0/3131262439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b8c075540 0x7f8b8c19b3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.592+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 >> 172.21.15.67:0/3131262439 conn(0x7f8b8c0fe830 msgr2=0x7f8b8c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.593+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 shutdown_connections 2024-09-10T22:48:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.593+0000 7f8b91600640 1 -- 172.21.15.67:0/3131262439 wait complete. 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.600+0000 7fb19de00640 1 -- 172.21.15.67:0/3358833625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 msgr2=0x7fb19810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.600+0000 7fb19de00640 1 --2- 172.21.15.67:0/3358833625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19810ec50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002a410 comp rx=0 tx=0).stop 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 -- 172.21.15.67:0/3358833625 shutdown_connections 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 --2- 172.21.15.67:0/3358833625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 0x7fb198111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 --2- 172.21.15.67:0/3358833625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19810ec50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 --2- 172.21.15.67:0/3358833625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 0x7fb198075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 -- 172.21.15.67:0/3358833625 >> 172.21.15.67:0/3358833625 conn(0x7fb1980fe6d0 msgr2=0x7fb198100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 -- 172.21.15.67:0/3358833625 shutdown_connections 2024-09-10T22:48:52.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.601+0000 7fb19de00640 1 -- 172.21.15.67:0/3358833625 wait complete. 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 Processor -- start 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 -- start start 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 0x7fb19819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 0x7fb1981a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb198114c80 con 0x7fb198075f40 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb198114b00 con 0x7fb19810f220 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb19de00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb198114e00 con 0x7fb198075540 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.602+0000 7fb18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36724/0 (socket says 172.21.15.67:36724) 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 -- 172.21.15.67:0/3997719473 learned_addr learned my addr 172.21.15.67:0/3997719473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb19ca00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 0x7fb19819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb19d400640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 0x7fb1981a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 -- 172.21.15.67:0/3997719473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 msgr2=0x7fb19819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 0x7fb19819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 -- 172.21.15.67:0/3997719473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 msgr2=0x7fb1981a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 0x7fb1981a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 -- 172.21.15.67:0/3997719473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1981a1240 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.603+0000 7fb18fe00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb180009230 tx=0x7fb180004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.604+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18003f030 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.604+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb180003a80 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.604+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180035070 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.604+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1981a14d0 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.604+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1981a7bf0 con 0x7fb198075f40 2024-09-10T22:48:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.605+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb198075920 con 0x7fb198075f40 2024-09-10T22:48:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.607+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fb1800035f0 con 0x7fb198075f40 2024-09-10T22:48:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.609+0000 7fb196000640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 0x7fb17c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.609+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb1801047a0 con 0x7fb198075f40 2024-09-10T22:48:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.609+0000 7fb19ca00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 0x7fb17c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.610+0000 7fb19ca00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 0x7fb17c0b7880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb188005fd0 tx=0x7fb18800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.613+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1800ccb70 con 0x7fb198075f40 2024-09-10T22:48:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 -- 172.21.15.67:0/2982437692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4102c10 msgr2=0x7f5bb4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 --2- 172.21.15.67:0/2982437692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4102c10 0x7f5bb4102ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009900 tx=0x7f5ba802a410 comp rx=0 tx=0).stop 2024-09-10T22:48:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 -- 172.21.15.67:0/2982437692 shutdown_connections 2024-09-10T22:48:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 --2- 172.21.15.67:0/2982437692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4108fa0 0x7f5bb410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 --2- 172.21.15.67:0/2982437692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 0x7f5bb4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 --2- 172.21.15.67:0/2982437692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4102c10 0x7f5bb4102ff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 -- 172.21.15.67:0/2982437692 >> 172.21.15.67:0/2982437692 conn(0x7f5bb40fe6c0 msgr2=0x7f5bb4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 -- 172.21.15.67:0/2982437692 shutdown_connections 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.269+0000 7f5bbb200640 1 -- 172.21.15.67:0/2982437692 wait complete. 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.270+0000 7f5bbb200640 1 Processor -- start 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.270+0000 7f5bbb200640 1 -- start start 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 0x7f5bb419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4112b20 con 0x7f5bb4108fa0 2024-09-10T22:48:52.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41129a0 con 0x7f5bb4102c10 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bbb200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4112ca0 con 0x7f5bb4108620 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bba800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bba800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36690/0 (socket says 172.21.15.67:36690) 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52646/0 (socket says 172.21.15.67:52646) 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bba800640 1 -- 172.21.15.67:0/3509456553 learned_addr learned my addr 172.21.15.67:0/3509456553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9400640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 0x7f5bb419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 -- 172.21.15.67:0/3509456553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 msgr2=0x7f5bb419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 0x7f5bb419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 -- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 msgr2=0x7f5bb41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bb9e00640 1 -- 172.21.15.67:0/3509456553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb41a1200 con 0x7f5bb4102c10 2024-09-10T22:48:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.271+0000 7f5bba800640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5bb9e00640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ba802a930 tx=0x7f5ba8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba803f070 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ba803baf0 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8035030 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb41a1490 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.272+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41a7bb0 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.273+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb4102ff0 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.274+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f5ba803b660 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.275+0000 7f5baea00640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 0x7f5b9c07c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.275+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5ba80bc7f0 con 0x7f5bb4102c10 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.275+0000 7f5bb9400640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 0x7f5b9c07c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.276+0000 7f5bb9400640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 0x7f5b9c07c960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4107ff0 tx=0x7f5ba400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.279+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ba8084d60 con 0x7f5bb4102c10 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.485+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5bb410f190 con 0x7f5bb4102c10 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.491+0000 7f5baea00640 1 -- 172.21.15.67:0/3509456553 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7f5ba8089790 con 0x7f5bb4102c10 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 msgr2=0x7f5b9c07c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 0x7f5b9c07c960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4107ff0 tx=0x7f5ba400b390 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 msgr2=0x7f5bb419b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ba802a930 tx=0x7f5ba8003b90 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 shutdown_connections 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b9c07a4a0 0x7f5b9c07c960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5bb4108fa0 0x7f5bb41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb4108620 0x7f5bb419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 --2- 172.21.15.67:0/3509456553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bb4102c10 0x7f5bb419b150 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.493+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 >> 172.21.15.67:0/3509456553 conn(0x7f5bb40fe6c0 msgr2=0x7f5bb4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:52.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.494+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 shutdown_connections 2024-09-10T22:48:52.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:51.494+0000 7f5bbb200640 1 -- 172.21.15.67:0/3509456553 wait complete. 2024-09-10T22:48:52.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:52 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:52.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:52 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3381587780' entity='client.admin' 2024-09-10T22:48:52.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:52 smithi092 ceph-mon[34223]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T22:48:52.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:48:52.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:52 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:52.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:52 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3381587780' entity='client.admin' 2024-09-10T22:48:52.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:52 smithi067 ceph-mon[30939]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T22:48:52.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:48:52.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.810+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fb1981a7f50 con 0x7fb198075f40 2024-09-10T22:48:52.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.815+0000 7fb196000640 1 -- 172.21.15.67:0/3997719473 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7fb1800d15a0 con 0x7fb198075f40 2024-09-10T22:48:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 msgr2=0x7fb17c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 0x7fb17c0b7880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb188005fd0 tx=0x7fb18800b390 comp rx=0 tx=0).stop 2024-09-10T22:48:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 msgr2=0x7fb19819b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb180009230 tx=0x7fb180004270 comp rx=0 tx=0).stop 2024-09-10T22:48:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 shutdown_connections 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb17c0b53c0 0x7fb17c0b7880 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19810f220 0x7fb1981a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb198075f40 0x7fb19819b6d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 --2- 172.21.15.67:0/3997719473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb198075540 0x7fb19819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 >> 172.21.15.67:0/3997719473 conn(0x7fb1980fe6d0 msgr2=0x7fb198111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 shutdown_connections 2024-09-10T22:48:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:52.817+0000 7fb19de00640 1 -- 172.21.15.67:0/3997719473 wait complete. 2024-09-10T22:48:52.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:52 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:52.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:52 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3381587780' entity='client.admin' 2024-09-10T22:48:52.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:52 smithi145 ceph-mon[34397]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T22:48:52.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:48:53.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:53 smithi067 ceph-mon[30939]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:53.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:53 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3997719473' entity='client.admin' 2024-09-10T22:48:53.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:53 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' 2024-09-10T22:48:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.015+0000 7f8525200640 1 -- 172.21.15.67:0/2933940668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f852010b330 msgr2=0x7f8520111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.015+0000 7f8525200640 1 --2- 172.21.15.67:0/2933940668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f852010b330 0x7f8520111b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8518009ca0 tx=0x7f851802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 -- 172.21.15.67:0/2933940668 shutdown_connections 2024-09-10T22:48:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 --2- 172.21.15.67:0/2933940668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f852010b330 0x7f8520111b30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 --2- 172.21.15.67:0/2933940668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010a9b0 0x7f852010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 --2- 172.21.15.67:0/2933940668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8520104fa0 0x7f8520105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 -- 172.21.15.67:0/2933940668 >> 172.21.15.67:0/2933940668 conn(0x7f8520100a50 msgr2=0x7f8520102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 -- 172.21.15.67:0/2933940668 shutdown_connections 2024-09-10T22:48:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.016+0000 7f8525200640 1 -- 172.21.15.67:0/2933940668 wait complete. 2024-09-10T22:48:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 Processor -- start 2024-09-10T22:48:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 -- start start 2024-09-10T22:48:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 0x7f852019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010b330 0x7f85201a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8520114e60 con 0x7f8520104fa0 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8520114ce0 con 0x7f852010a9b0 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8525200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8520114fe0 con 0x7f852010b330 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f851de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f851de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36768/0 (socket says 172.21.15.67:36768) 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f851de00640 1 -- 172.21.15.67:0/483121403 learned_addr learned my addr 172.21.15.67:0/483121403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.017+0000 7f8517e00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 0x7f852019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851e800640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010b330 0x7f85201a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 -- 172.21.15.67:0/483121403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010b330 msgr2=0x7f85201a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010b330 0x7f85201a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 -- 172.21.15.67:0/483121403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 msgr2=0x7f852019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 0x7f852019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 -- 172.21.15.67:0/483121403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85201a1420 con 0x7f8520104fa0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f8517e00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 0x7f852019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f851de00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9f0 tx=0x7f851000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.018+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000cd80 con 0x7f8520104fa0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.019+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85100108c0 con 0x7f8520104fa0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.019+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85201a1710 con 0x7f8520104fa0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.019+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100187c0 con 0x7f8520104fa0 2024-09-10T22:48:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.019+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85201a7dd0 con 0x7f8520104fa0 2024-09-10T22:48:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.020+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8520105380 con 0x7f8520104fa0 2024-09-10T22:48:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.022+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f8510010430 con 0x7f8520104fa0 2024-09-10T22:48:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.023+0000 7f8516a00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 0x7f85040b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.024+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f85100e4410 con 0x7f8520104fa0 2024-09-10T22:48:54.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.024+0000 7f8517e00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 0x7f85040b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.024+0000 7f8517e00640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 0x7f85040b7760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8520107410 tx=0x7f8508005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.027+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f85100ac7e0 con 0x7f8520104fa0 2024-09-10T22:48:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:53 smithi145 ceph-mon[34397]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:53 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3997719473' entity='client.admin' 2024-09-10T22:48:54.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:53 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' 2024-09-10T22:48:54.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 -- 172.21.15.67:0/2717486112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c0754e0 msgr2=0x7fd84c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 --2- 172.21.15.67:0/2717486112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c0754e0 0x7fd84c0758c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009900 tx=0x7fd83c02a410 comp rx=0 tx=0).stop 2024-09-10T22:48:54.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 -- 172.21.15.67:0/2717486112 shutdown_connections 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 --2- 172.21.15.67:0/2717486112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 --2- 172.21.15.67:0/2717486112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c075e00 0x7fd84c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 --2- 172.21.15.67:0/2717486112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c0754e0 0x7fd84c0758c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.970+0000 7fd851200640 1 -- 172.21.15.67:0/2717486112 >> 172.21.15.67:0/2717486112 conn(0x7fd84c0fe830 msgr2=0x7fd84c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.971+0000 7fd851200640 1 -- 172.21.15.67:0/2717486112 shutdown_connections 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.971+0000 7fd851200640 1 -- 172.21.15.67:0/2717486112 wait complete. 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.971+0000 7fd851200640 1 Processor -- start 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 -- start start 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c075e00 0x7fd84c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c114e80 con 0x7fd84c10f3b0 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c114d00 con 0x7fd84c0754e0 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd851200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c115000 con 0x7fd84c075e00 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd84a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd849e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd849e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52604/0 (socket says 172.21.15.67:52604) 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd849e00640 1 -- 172.21.15.67:0/1373989339 learned_addr learned my addr 172.21.15.67:0/1373989339 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.972+0000 7fd843e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c075e00 0x7fd84c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 -- 172.21.15.67:0/1373989339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c075e00 msgr2=0x7fd84c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c075e00 0x7fd84c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 -- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 msgr2=0x7fd84c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 -- 172.21.15.67:0/1373989339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c1a1460 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd84a800640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd849e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02a930 tx=0x7fd83c003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c03f070 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a16f0 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd83c003e40 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c035030 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.973+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a7e10 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.975+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c0758c0 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.976+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7fd83c031400 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.977+0000 7fd842a00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 0x7fd83007c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.977+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd83c0bca40 con 0x7fd84c0754e0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.977+0000 7fd843e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 0x7fd83007c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.977+0000 7fd843e00640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 0x7fd83007c8c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd84c111a20 tx=0x7fd83400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:49.985+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd83c084f30 con 0x7fd84c0754e0 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.183+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd84c1a80a0 con 0x7fd84c0754e0 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.190+0000 7fd842a00640 1 -- 172.21.15.67:0/1373989339 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v23)=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7fd83c089960 con 0x7fd84c0754e0 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 msgr2=0x7fd83007c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 0x7fd83007c8c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd84c111a20 tx=0x7fd83400b410 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 msgr2=0x7fd84c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02a930 tx=0x7fd83c003c20 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 shutdown_connections 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd83007a400 0x7fd83007c8c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd84c10f3b0 0x7fd84c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c075e00 0x7fd84c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 --2- 172.21.15.67:0/1373989339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd84c0754e0 0x7fd84c19b3b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 >> 172.21.15.67:0/1373989339 conn(0x7fd84c0fe830 msgr2=0x7fd84c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 shutdown_connections 2024-09-10T22:48:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:50.193+0000 7fd851200640 1 -- 172.21.15.67:0/1373989339 wait complete. 2024-09-10T22:48:54.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.224+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f852002e5c0 con 0x7f8520104fa0 2024-09-10T22:48:54.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.228+0000 7f8516a00640 1 -- 172.21.15.67:0/483121403 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f85100b1210 con 0x7f8520104fa0 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.230+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 msgr2=0x7f85040b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.230+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 0x7f85040b7760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8520107410 tx=0x7f8508005f90 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.230+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 msgr2=0x7f852019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.230+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9f0 tx=0x7f851000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 shutdown_connections 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85040b52a0 0x7f85040b7760 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f852010b330 0x7f85201a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852010a9b0 0x7f852019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 --2- 172.21.15.67:0/483121403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520104fa0 0x7f852019b370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 >> 172.21.15.67:0/483121403 conn(0x7f8520100a50 msgr2=0x7f8520102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 shutdown_connections 2024-09-10T22:48:54.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.231+0000 7f8525200640 1 -- 172.21.15.67:0/483121403 wait complete. 2024-09-10T22:48:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:53 smithi092 ceph-mon[34223]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:53 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3997719473' entity='client.admin' 2024-09-10T22:48:54.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:53 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' 2024-09-10T22:48:55.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:55 smithi092 ceph-mon[34223]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:55.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:55 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/483121403' entity='client.admin' 2024-09-10T22:48:55.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:55 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3914229704' entity='client.admin' 2024-09-10T22:48:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:55 smithi067 ceph-mon[30939]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:55 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/483121403' entity='client.admin' 2024-09-10T22:48:55.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:55 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3914229704' entity='client.admin' 2024-09-10T22:48:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:55 smithi145 ceph-mon[34397]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:55 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/483121403' entity='client.admin' 2024-09-10T22:48:55.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:55 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3914229704' entity='client.admin' 2024-09-10T22:48:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 -- 172.21.15.67:0/1637993920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 msgr2=0x7f70b4111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 --2- 172.21.15.67:0/1637993920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b4111b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009900 tx=0x7f70ac02a410 comp rx=0 tx=0).stop 2024-09-10T22:48:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 -- 172.21.15.67:0/1637993920 shutdown_connections 2024-09-10T22:48:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 --2- 172.21.15.67:0/1637993920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b4111b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 --2- 172.21.15.67:0/1637993920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 0x7f70b410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 --2- 172.21.15.67:0/1637993920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b41053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 -- 172.21.15.67:0/1637993920 >> 172.21.15.67:0/1637993920 conn(0x7f70b41009f0 msgr2=0x7f70b4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 -- 172.21.15.67:0/1637993920 shutdown_connections 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.573+0000 7f70bc200640 1 -- 172.21.15.67:0/1637993920 wait complete. 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 Processor -- start 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 -- start start 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 0x7f70b419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4114e80 con 0x7f70b410b350 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4114d00 con 0x7f70b410a9d0 2024-09-10T22:48:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.574+0000 7f70bc200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4115000 con 0x7f70b4104fc0 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70ba200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70ba200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36742/0 (socket says 172.21.15.67:36742) 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70ba200640 1 -- 172.21.15.67:0/351961481 learned_addr learned my addr 172.21.15.67:0/351961481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b8e00640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 0x7f70b419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 -- 172.21.15.67:0/351961481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 msgr2=0x7f70b419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 0x7f70b419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 -- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 msgr2=0x7f70b41a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70b9800640 1 -- 172.21.15.67:0/351961481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b41a1420 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.575+0000 7f70ba200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70b9800640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b419b370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009fd0 tx=0x7f70a400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004b20 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b41a1710 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70a4004cc0 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4010690 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.576+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41a7dd0 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.577+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b41053a0 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.579+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f70a4046ac0 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.581+0000 7f70b2000640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 0x7f70980b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.581+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f70a40e4790 con 0x7f70b4104fc0 2024-09-10T22:48:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.581+0000 7f70b8e00640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 0x7f70980b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.582+0000 7f70b8e00640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 0x7f70980b7760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70b4104290 tx=0x7f70a000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.583+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f70a40ad1b0 con 0x7f70b4104fc0 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.780+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f70b402e5c0 con 0x7f70b4104fc0 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.787+0000 7f70b2000640 1 -- 172.21.15.67:0/351961481 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v28)=0 v28) ==== 120+0+0 (secure 0 0 0) 0x7f70a40a8030 con 0x7f70b4104fc0 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 msgr2=0x7f70980b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 0x7f70980b7760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70b4104290 tx=0x7f70a000b390 comp rx=0 tx=0).stop 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 msgr2=0x7f70b419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b419b370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009fd0 tx=0x7f70a400ef90 comp rx=0 tx=0).stop 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 shutdown_connections 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f70980b52a0 0x7f70980b7760 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70b410b350 0x7f70b41a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70b410a9d0 0x7f70b419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 --2- 172.21.15.67:0/351961481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70b4104fc0 0x7f70b419b370 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 >> 172.21.15.67:0/351961481 conn(0x7f70b41009f0 msgr2=0x7f70b4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 shutdown_connections 2024-09-10T22:48:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:53.790+0000 7f70bc200640 1 -- 172.21.15.67:0/351961481 wait complete. 2024-09-10T22:48:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 -- 172.21.15.67:0/865895197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400109120 msgr2=0x7fa40010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 --2- 172.21.15.67:0/865895197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400109120 0x7fa40010f920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009ca0 tx=0x7fa3f802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 -- 172.21.15.67:0/865895197 shutdown_connections 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 --2- 172.21.15.67:0/865895197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400109120 0x7fa40010f920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 --2- 172.21.15.67:0/865895197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4001087a0 0x7fa400108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 --2- 172.21.15.67:0/865895197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa400102d90 0x7fa400103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 -- 172.21.15.67:0/865895197 >> 172.21.15.67:0/865895197 conn(0x7fa4000fe820 msgr2=0x7fa400100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.500+0000 7fa404a00640 1 -- 172.21.15.67:0/865895197 shutdown_connections 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.501+0000 7fa404a00640 1 -- 172.21.15.67:0/865895197 wait complete. 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.501+0000 7fa404a00640 1 Processor -- start 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 -- start start 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4001087a0 0x7fa40019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400109120 0x7fa4001a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa400112ce0 con 0x7fa400102d90 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa400112b60 con 0x7fa400109120 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa404a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa400112e60 con 0x7fa4001087a0 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa3fde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa3fde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36806/0 (socket says 172.21.15.67:36806) 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa3fde00640 1 -- 172.21.15.67:0/1632653539 learned_addr learned my addr 172.21.15.67:0/1632653539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa3fe800640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400109120 0x7fa4001a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.502+0000 7fa3fd400640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4001087a0 0x7fa40019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 -- 172.21.15.67:0/1632653539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4001087a0 msgr2=0x7fa40019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4001087a0 0x7fa40019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 -- 172.21.15.67:0/1632653539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400109120 msgr2=0x7fa4001a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400109120 0x7fa4001a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 -- 172.21.15.67:0/1632653539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4001a13d0 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.503+0000 7fa3fde00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d950 tx=0x7fa3f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.504+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0017030 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.504+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3f0004020 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.504+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4001a16c0 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.504+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00055d0 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.504+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4001a7f50 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.505+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa400103170 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.507+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fa3f0009a30 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.510+0000 7fa3f6a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 0x7fa3e40b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.510+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa3f00e39e0 con 0x7fa400102d90 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.510+0000 7fa3fd400640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 0x7fa3e40b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.511+0000 7fa3fd400640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 0x7fa3e40b7860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa4001020d0 tx=0x7fa3ec00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.513+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa3f00ac400 con 0x7fa400102d90 2024-09-10T22:48:56.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.710+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa40002e5c0 con 0x7fa400102d90 2024-09-10T22:48:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.714+0000 7fa3f6a00640 1 -- 172.21.15.67:0/1632653539 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7fa3f00abd00 con 0x7fa400102d90 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 msgr2=0x7fa3e40b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 0x7fa3e40b7860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa4001020d0 tx=0x7fa3ec00b410 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 msgr2=0x7fa40019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d950 tx=0x7fa3f000de10 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 shutdown_connections 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa3e40b53a0 0x7fa3e40b7860 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa400109120 0x7fa4001a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4001087a0 0x7fa40019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 --2- 172.21.15.67:0/1632653539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa400102d90 0x7fa40019b320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 >> 172.21.15.67:0/1632653539 conn(0x7fa4000fe820 msgr2=0x7fa400104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 shutdown_connections 2024-09-10T22:48:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:56.717+0000 7fa404a00640 1 -- 172.21.15.67:0/1632653539 wait complete. 2024-09-10T22:48:57.390 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:57 smithi092 ceph-mon[34223]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:57 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1632653539' entity='client.admin' 2024-09-10T22:48:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:57 smithi067 ceph-mon[30939]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:57 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1632653539' entity='client.admin' 2024-09-10T22:48:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:57 smithi145 ceph-mon[34397]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:57 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1632653539' entity='client.admin' 2024-09-10T22:48:59.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 -- 172.21.15.67:0/3221165165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 msgr2=0x7f14ec111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 --2- 172.21.15.67:0/3221165165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec111980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009ca0 tx=0x7f14e402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 -- 172.21.15.67:0/3221165165 shutdown_connections 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 --2- 172.21.15.67:0/3221165165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec111980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 --2- 172.21.15.67:0/3221165165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec10a800 0x7f14ec10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 --2- 172.21.15.67:0/3221165165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec104df0 0x7f14ec1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 -- 172.21.15.67:0/3221165165 >> 172.21.15.67:0/3221165165 conn(0x7f14ec100840 msgr2=0x7f14ec102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 -- 172.21.15.67:0/3221165165 shutdown_connections 2024-09-10T22:48:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.113+0000 7f14eb200640 1 -- 172.21.15.67:0/3221165165 wait complete. 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.114+0000 7f14eb200640 1 Processor -- start 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.114+0000 7f14eb200640 1 -- start start 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.114+0000 7f14eb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec104df0 0x7f14ec19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.114+0000 7f14eb200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec10a800 0x7f14ec19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14eb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14eb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec114d40 con 0x7f14ec10b180 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14eb200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec114bc0 con 0x7f14ec10a800 2024-09-10T22:48:59.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14eb200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec114ec0 con 0x7f14ec104df0 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43920/0 (socket says 172.21.15.67:43920) 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 -- 172.21.15.67:0/544509973 learned_addr learned my addr 172.21.15.67:0/544509973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14e9e00640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec104df0 0x7f14ec19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14e9400640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec10a800 0x7f14ec19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 -- 172.21.15.67:0/544509973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec104df0 msgr2=0x7f14ec19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec104df0 0x7f14ec19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 -- 172.21.15.67:0/544509973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec10a800 msgr2=0x7f14ec19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec10a800 0x7f14ec19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.115+0000 7f14ea800640 1 -- 172.21.15.67:0/544509973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec1a1130 con 0x7f14ec10b180 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14ea800640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f14e4031040 tx=0x7f14e4003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4032c90 con 0x7f14ec10b180 2024-09-10T22:48:59.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14e4032e30 con 0x7f14ec10b180 2024-09-10T22:48:59.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4030af0 con 0x7f14ec10b180 2024-09-10T22:48:59.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec1a13c0 con 0x7f14ec10b180 2024-09-10T22:48:59.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.116+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec1a7ae0 con 0x7f14ec10b180 2024-09-10T22:48:59.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.117+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec1051d0 con 0x7f14ec10b180 2024-09-10T22:48:59.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.120+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f14e4030c90 con 0x7f14ec10b180 2024-09-10T22:48:59.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.121+0000 7f14e2a00640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 0x7f14d00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.121+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f14e41051e0 con 0x7f14ec10b180 2024-09-10T22:48:59.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.121+0000 7f14e9e00640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 0x7f14d00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.122+0000 7f14e9e00640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 0x7f14d00b7880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009bd0 tx=0x7f14dc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.123+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f14e40cdc00 con 0x7f14ec10b180 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 -- 172.21.15.67:0/2315350694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 msgr2=0x7fbf7c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 --2- 172.21.15.67:0/2315350694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c105380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009900 tx=0x7fbf6c02a410 comp rx=0 tx=0).stop 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 -- 172.21.15.67:0/2315350694 shutdown_connections 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 --2- 172.21.15.67:0/2315350694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10b330 0x7fbf7c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 --2- 172.21.15.67:0/2315350694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10a9b0 0x7fbf7c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 --2- 172.21.15.67:0/2315350694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c105380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 -- 172.21.15.67:0/2315350694 >> 172.21.15.67:0/2315350694 conn(0x7fbf7c100a50 msgr2=0x7fbf7c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 -- 172.21.15.67:0/2315350694 shutdown_connections 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.993+0000 7fbf82e00640 1 -- 172.21.15.67:0/2315350694 wait complete. 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.994+0000 7fbf82e00640 1 Processor -- start 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 -- start start 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10a9b0 0x7fbf7c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10b330 0x7fbf7c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c114df0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c114c70 con 0x7fbf7c10b330 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf82e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c114f70 con 0x7fbf7c10a9b0 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36794/0 (socket says 172.21.15.67:36794) 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.995+0000 7fbf7be00640 1 -- 172.21.15.67:0/3914229704 learned_addr learned my addr 172.21.15.67:0/3914229704 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf80e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10b330 0x7fbf7c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7b400640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10a9b0 0x7fbf7c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 -- 172.21.15.67:0/3914229704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10a9b0 msgr2=0x7fbf7c19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10a9b0 0x7fbf7c19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 -- 172.21.15.67:0/3914229704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10b330 msgr2=0x7fbf7c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10b330 0x7fbf7c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 -- 172.21.15.67:0/3914229704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c1a14f0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf7be00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009230 tx=0x7fbf6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.996+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c03f030 con 0x7fbf7c104fa0 2024-09-10T22:48:59.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.997+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf6c003a80 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.997+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c1a1780 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.997+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c035070 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.997+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c1a7ea0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:54.998+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c105380 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.000+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fbf6c03bc30 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.001+0000 7fbf78c00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 0x7fbf600b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.002+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbf6c104f90 con 0x7fbf7c104fa0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.002+0000 7fbf7b400640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 0x7fbf600b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.002+0000 7fbf7b400640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 0x7fbf600b7760 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c10a670 tx=0x7fbf6800b590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:59.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.007+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbf6c0cd9b0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.213+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fbf7c02e5c0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.217+0000 7fbf78c00640 1 -- 172.21.15.67:0/3914229704 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7fbf6c0cd2b0 con 0x7fbf7c104fa0 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.219+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 msgr2=0x7fbf600b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.219+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 0x7fbf600b7760 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c10a670 tx=0x7fbf6800b590 comp rx=0 tx=0).stop 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 msgr2=0x7fbf7c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009230 tx=0x7fbf6c004270 comp rx=0 tx=0).stop 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 shutdown_connections 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbf600b52a0 0x7fbf600b7760 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf7c10b330 0x7fbf7c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7c10a9b0 0x7fbf7c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 --2- 172.21.15.67:0/3914229704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf7c104fa0 0x7fbf7c19b440 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 >> 172.21.15.67:0/3914229704 conn(0x7fbf7c100a50 msgr2=0x7fbf7c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 shutdown_connections 2024-09-10T22:48:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:55.220+0000 7fbf82e00640 1 -- 172.21.15.67:0/3914229704 wait complete. 2024-09-10T22:48:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.321+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f14ec1a7ed0 con 0x7f14ec10b180 2024-09-10T22:48:59.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.325+0000 7f14e2a00640 1 -- 172.21.15.67:0/544509973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v32)=0 v32) ==== 127+0+0 (secure 0 0 0) 0x7f14e40cd500 con 0x7f14ec10b180 2024-09-10T22:48:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 msgr2=0x7f14d00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 0x7f14d00b7880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009bd0 tx=0x7f14dc005f90 comp rx=0 tx=0).stop 2024-09-10T22:48:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 msgr2=0x7f14ec1a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f14e4031040 tx=0x7f14e4003f80 comp rx=0 tx=0).stop 2024-09-10T22:48:59.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 shutdown_connections 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f14d00b53c0 0x7f14d00b7880 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14ec10b180 0x7f14ec1a0960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14ec10a800 0x7f14ec19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 --2- 172.21.15.67:0/544509973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14ec104df0 0x7f14ec19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 >> 172.21.15.67:0/544509973 conn(0x7f14ec100840 msgr2=0x7f14ec103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 shutdown_connections 2024-09-10T22:48:59.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:48:59.328+0000 7f14eb200640 1 -- 172.21.15.67:0/544509973 wait complete. 2024-09-10T22:48:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:59 smithi067 ceph-mon[30939]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:48:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:48:59 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/544509973' entity='client.admin' 2024-09-10T22:49:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:59 smithi092 ceph-mon[34223]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:48:59 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/544509973' entity='client.admin' 2024-09-10T22:49:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:59 smithi145 ceph-mon[34397]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:48:59 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/544509973' entity='client.admin' 2024-09-10T22:49:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:01 smithi067 ceph-mon[30939]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:01.859 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi067') 2024-09-10T22:49:01.860 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi067') 2024-09-10T22:49:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:01 smithi092 ceph-mon[34223]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:01 smithi145 ceph-mon[34397]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.313+0000 7fd022a00640 1 -- 172.21.15.67:0/2612043830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 msgr2=0x7fd01c075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.313+0000 7fd022a00640 1 --2- 172.21.15.67:0/2612043830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c075920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009900 tx=0x7fd00c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 -- 172.21.15.67:0/2612043830 shutdown_connections 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 --2- 172.21.15.67:0/2612043830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c113b20 0x7fd01c115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 --2- 172.21.15.67:0/2612043830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c075e60 0x7fd01c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 --2- 172.21.15.67:0/2612043830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c075920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 -- 172.21.15.67:0/2612043830 >> 172.21.15.67:0/2612043830 conn(0x7fd01c0fe830 msgr2=0x7fd01c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:02.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 -- 172.21.15.67:0/2612043830 shutdown_connections 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.314+0000 7fd022a00640 1 -- 172.21.15.67:0/2612043830 wait complete. 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.315+0000 7fd022a00640 1 Processor -- start 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.315+0000 7fd022a00640 1 -- start start 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 0x7fd01c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c113b20 0x7fd01c1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1192c0 con 0x7fd01c075540 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c119140 con 0x7fd01c075e60 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd022a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c119440 con 0x7fd01c113b20 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:02.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43944/0 (socket says 172.21.15.67:43944) 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 -- 172.21.15.67:0/3050755573 learned_addr learned my addr 172.21.15.67:0/3050755573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd020a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c113b20 0x7fd01c1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01b400640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 0x7fd01c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 -- 172.21.15.67:0/3050755573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c113b20 msgr2=0x7fd01c1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c113b20 0x7fd01c1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 -- 172.21.15.67:0/3050755573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 msgr2=0x7fd01c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 0x7fd01c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01be00640 1 -- 172.21.15.67:0/3050755573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c1a5900 con 0x7fd01c075540 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.316+0000 7fd01b400640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 0x7fd01c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:02.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd01be00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd00c02a910 tx=0x7fd00c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c03f070 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd00c003e40 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c035030 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c1a5b90 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.317+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1ac2b0 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.318+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01c075920 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.320+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fd00c031400 con 0x7fd01c075540 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.323+0000 7fd018c00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 0x7fd0000b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.323+0000 7fd01b400640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 0x7fd0000b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:02.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.323+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd00c104f60 con 0x7fd01c075540 2024-09-10T22:49:02.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.324+0000 7fd01b400640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 0x7fd0000b77e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd01c115060 tx=0x7fd008005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:02.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.325+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd00c0cd980 con 0x7fd01c075540 2024-09-10T22:49:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.525+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fd01c1ac550 con 0x7fd01c075540 2024-09-10T22:49:02.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.529+0000 7fd018c00640 1 -- 172.21.15.67:0/3050755573 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v33)=0 v33) ==== 115+0+0 (secure 0 0 0) 0x7fd00c0cd280 con 0x7fd01c075540 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 msgr2=0x7fd0000b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 0x7fd0000b77e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd01c115060 tx=0x7fd008005e50 comp rx=0 tx=0).stop 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 msgr2=0x7fd01c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd00c02a910 tx=0x7fd00c003c20 comp rx=0 tx=0).stop 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 shutdown_connections 2024-09-10T22:49:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd0000b5320 0x7fd0000b77e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd01c113b20 0x7fd01c1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01c075e60 0x7fd01c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 --2- 172.21.15.67:0/3050755573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd01c075540 0x7fd01c19f850 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 >> 172.21.15.67:0/3050755573 conn(0x7fd01c0fe830 msgr2=0x7fd01c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 shutdown_connections 2024-09-10T22:49:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:02.532+0000 7fd022a00640 1 -- 172.21.15.67:0/3050755573 wait complete. 2024-09-10T22:49:03.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:03 smithi067 ceph-mon[30939]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:03.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:03 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3050755573' entity='client.admin' 2024-09-10T22:49:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:03 smithi145 ceph-mon[34397]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:03.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3050755573' entity='client.admin' 2024-09-10T22:49:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:03 smithi092 ceph-mon[34223]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:03 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3050755573' entity='client.admin' 2024-09-10T22:49:04.789 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.789 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.789 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.789 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi067') 2024-09-10T22:49:04.790 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi067') 2024-09-10T22:49:05.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 -- 172.21.15.67:0/2134199998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010b350 msgr2=0x7fdae0111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 --2- 172.21.15.67:0/2134199998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010b350 0x7fdae0111b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 -- 172.21.15.67:0/2134199998 shutdown_connections 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 --2- 172.21.15.67:0/2134199998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010b350 0x7fdae0111b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 --2- 172.21.15.67:0/2134199998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae010a9d0 0x7fdae010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 --2- 172.21.15.67:0/2134199998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae0104fc0 0x7fdae01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 -- 172.21.15.67:0/2134199998 >> 172.21.15.67:0/2134199998 conn(0x7fdae01009f0 msgr2=0x7fdae0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 -- 172.21.15.67:0/2134199998 shutdown_connections 2024-09-10T22:49:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.459+0000 7fdae5a00640 1 -- 172.21.15.67:0/2134199998 wait complete. 2024-09-10T22:49:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.460+0000 7fdae5a00640 1 Processor -- start 2024-09-10T22:49:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.460+0000 7fdae5a00640 1 -- start start 2024-09-10T22:49:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 0x7fdae019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 0x7fdae01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0114e80 con 0x7fdae010a9d0 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0114d00 con 0x7fdae0104fc0 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdae5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0115000 con 0x7fdae010b350 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdade000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdade000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43996/0 (socket says 172.21.15.67:43996) 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdade000640 1 -- 172.21.15.67:0/1301844584 learned_addr learned my addr 172.21.15.67:0/1301844584 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdadf400640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 0x7fdae01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.461+0000 7fdadea00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 0x7fdae019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 -- 172.21.15.67:0/1301844584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 msgr2=0x7fdae01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 0x7fdae01a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 -- 172.21.15.67:0/1301844584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 msgr2=0x7fdae019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 0x7fdae019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 -- 172.21.15.67:0/1301844584 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae01a1420 con 0x7fdae010a9d0 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdadf400640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 0x7fdae01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdadea00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 0x7fdae019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdade000640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d950 tx=0x7fdacc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017030 con 0x7fdae010a9d0 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdacc004020 con 0x7fdae010a9d0 2024-09-10T22:49:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae01a1710 con 0x7fdae010a9d0 2024-09-10T22:49:05.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.462+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0055d0 con 0x7fdae010a9d0 2024-09-10T22:49:05.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.463+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a7dd0 con 0x7fdae010a9d0 2024-09-10T22:49:05.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.464+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae01053a0 con 0x7fdae010a9d0 2024-09-10T22:49:05.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.469+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fdacc009a30 con 0x7fdae010a9d0 2024-09-10T22:49:05.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.471+0000 7fdad7400640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 0x7fdac40b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.471+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdacc0e40c0 con 0x7fdae010a9d0 2024-09-10T22:49:05.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.471+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdacc0e44f0 con 0x7fdae010a9d0 2024-09-10T22:49:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.471+0000 7fdadea00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 0x7fdac40b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.472+0000 7fdadea00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 0x7fdac40b7760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009c20 tx=0x7fdad0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:05.595 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:05 smithi067 ceph-mon[30939]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:05 smithi145 ceph-mon[34397]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:05.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.697+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fdae019c4a0 con 0x7fdae010a9d0 2024-09-10T22:49:05.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.702+0000 7fdad7400640 1 -- 172.21.15.67:0/1301844584 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) ==== 138+0+0 (secure 0 0 0) 0x7fdacc0acae0 con 0x7fdae010a9d0 2024-09-10T22:49:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 msgr2=0x7fdac40b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 0x7fdac40b7760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009c20 tx=0x7fdad0005f90 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 msgr2=0x7fdae019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d950 tx=0x7fdacc00de10 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 shutdown_connections 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fdac40b52a0 0x7fdac40b7760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdae010b350 0x7fdae01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae010a9d0 0x7fdae019b8b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 --2- 172.21.15.67:0/1301844584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdae0104fc0 0x7fdae019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.704+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 >> 172.21.15.67:0/1301844584 conn(0x7fdae01009f0 msgr2=0x7fdae0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.705+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 shutdown_connections 2024-09-10T22:49:05.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.705+0000 7fdae5a00640 1 -- 172.21.15.67:0/1301844584 wait complete. 2024-09-10T22:49:05.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 -- 172.21.15.67:0/341069456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 msgr2=0x7f5f24101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 --2- 172.21.15.67:0/341069456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24101ee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009860 tx=0x7f5f1802a340 comp rx=0 tx=0).stop 2024-09-10T22:49:05.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 -- 172.21.15.67:0/341069456 shutdown_connections 2024-09-10T22:49:05.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 --2- 172.21.15.67:0/341069456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107e90 0x7f5f2410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 --2- 172.21.15.67:0/341069456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107510 0x7f5f24107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 --2- 172.21.15.67:0/341069456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24101ee0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 -- 172.21.15.67:0/341069456 >> 172.21.15.67:0/341069456 conn(0x7f5f240fd5b0 msgr2=0x7f5f240ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:05.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 -- 172.21.15.67:0/341069456 shutdown_connections 2024-09-10T22:49:05.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.355+0000 7f5f2b200640 1 -- 172.21.15.67:0/341069456 wait complete. 2024-09-10T22:49:05.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.356+0000 7f5f2b200640 1 Processor -- start 2024-09-10T22:49:05.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 -- start start 2024-09-10T22:49:05.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107510 0x7f5f24198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 0x7f5f241985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24111a00 con 0x7f5f24107e90 2024-09-10T22:49:05.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24111880 con 0x7f5f24107510 2024-09-10T22:49:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24111b80 con 0x7f5f24101b00 2024-09-10T22:49:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f29e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f2a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 0x7f5f241985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f29e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:56810/0 (socket says 172.21.15.67:56810) 2024-09-10T22:49:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f29e00640 1 -- 172.21.15.67:0/4085646605 learned_addr learned my addr 172.21.15.67:0/4085646605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:49:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.357+0000 7f5f29400640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107510 0x7f5f24198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 -- 172.21.15.67:0/4085646605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 msgr2=0x7f5f24197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 -- 172.21.15.67:0/4085646605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107510 msgr2=0x7f5f24198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107510 0x7f5f24198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 -- 172.21.15.67:0/4085646605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f241a7320 con 0x7f5f24107e90 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2a800640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 0x7f5f241985c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ef00 tx=0x7f5f2000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004b20 con 0x7f5f24107e90 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f20003e00 con 0x7f5f24107e90 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f241a7610 con 0x7f5f24107e90 2024-09-10T22:49:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.358+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004820 con 0x7f5f24107e90 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.359+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a7b50 con 0x7f5f24107e90 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.360+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f24101ee0 con 0x7f5f24107e90 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.362+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f5f2000b040 con 0x7f5f24107e90 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.363+0000 7f5f1ea00640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 0x7f5f0c0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.363+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5f200e3e40 con 0x7f5f24107e90 2024-09-10T22:49:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.364+0000 7f5f29e00640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 0x7f5f0c0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.364+0000 7f5f29e00640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 0x7f5f0c0b7800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005cd0 tx=0x7f5f180058b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.367+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5f20054220 con 0x7f5f24107e90 2024-09-10T22:49:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.571+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f5f2402e5c0 con 0x7f5f24107e90 2024-09-10T22:49:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.575+0000 7f5f1ea00640 1 -- 172.21.15.67:0/4085646605 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f5f20004cc0 con 0x7f5f24107e90 2024-09-10T22:49:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 msgr2=0x7f5f0c0b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 0x7f5f0c0b7800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005cd0 tx=0x7f5f180058b0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 msgr2=0x7f5f241985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 0x7f5f241985c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ef00 tx=0x7f5f2000c740 comp rx=0 tx=0).stop 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 shutdown_connections 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5f0c0b5340 0x7f5f0c0b7800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24107e90 0x7f5f241985c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f24107510 0x7f5f24198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 --2- 172.21.15.67:0/4085646605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f24101b00 0x7f5f24197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 >> 172.21.15.67:0/4085646605 conn(0x7f5f240fd5b0 msgr2=0x7f5f240ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.578+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 shutdown_connections 2024-09-10T22:49:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:05.579+0000 7f5f2b200640 1 -- 172.21.15.67:0/4085646605 wait complete. 2024-09-10T22:49:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:05 smithi092 ceph-mon[34223]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:06.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4085646605' entity='client.admin' 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1301844584' entity='client.admin' 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:06.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:06.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4085646605' entity='client.admin' 2024-09-10T22:49:06.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1301844584' entity='client.admin' 2024-09-10T22:49:06.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:06.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:06.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:06.963 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4085646605' entity='client.admin' 2024-09-10T22:49:06.963 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1301844584' entity='client.admin' 2024-09-10T22:49:06.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:06.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:06.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:06.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:07.281 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi067') 2024-09-10T22:49:07.281 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi067') 2024-09-10T22:49:07.281 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi067') 2024-09-10T22:49:07.282 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi067') 2024-09-10T22:49:07.282 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5c4a3ee4-6fc6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-09-10T22:49:07.282 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T22:49:07.283 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:49:07.283 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:49:07.319 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:49:07.320 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-09-10T22:49:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:49:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-10 22:47:33.771731027 +0000 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-10 22:42:59.835035441 +0000 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-10 22:42:59.835035441 +0000 2024-09-10T22:49:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-10 22:42:59.835035441 +0000 2024-09-10T22:49:07.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:49:07.490 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-10T22:49:07.491 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-10T22:49:07.491 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000500716 s, 1.0 MB/s 2024-09-10T22:49:07.493 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:49:07.530 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-10 22:47:33.810730176 +0000 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-10 22:43:00.324024717 +0000 2024-09-10T22:49:07.606 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-10 22:43:00.324024717 +0000 2024-09-10T22:49:07.607 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-10 22:43:00.324024717 +0000 2024-09-10T22:49:07.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:49:07.699 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-10T22:49:07.699 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-10T22:49:07.699 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000288488 s, 1.8 MB/s 2024-09-10T22:49:07.701 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:49:07.738 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-09-10T22:49:07.769 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:07 smithi092 ceph-mon[34223]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:07.769 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:07 smithi092 ceph-mon[34223]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:49:07.770 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:07 smithi092 ceph-mon[34223]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:49:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:07 smithi067 ceph-mon[30939]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:07 smithi067 ceph-mon[30939]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:49:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:07 smithi067 ceph-mon[30939]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:49:07.813 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:49:07.814 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:07.814 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-10T22:49:07.814 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:07.814 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:07.814 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-10 22:47:33.857729150 +0000 2024-09-10T22:49:07.815 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-10 22:43:00.807014125 +0000 2024-09-10T22:49:07.815 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-10 22:43:00.807014125 +0000 2024-09-10T22:49:07.815 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-10 22:43:00.807014125 +0000 2024-09-10T22:49:07.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:49:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:07 smithi145 ceph-mon[34397]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:07 smithi145 ceph-mon[34397]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:49:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:07 smithi145 ceph-mon[34397]: Reconfiguring daemon mon.b on smithi092 2024-09-10T22:49:07.906 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-10T22:49:07.906 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-10T22:49:07.906 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000295975 s, 1.7 MB/s 2024-09-10T22:49:07.909 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:49:07.945 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-09-10T22:49:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-10 22:47:33.901728189 +0000 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-10 22:43:01.273003905 +0000 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-10 22:43:01.273003905 +0000 2024-09-10T22:49:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-10 22:43:01.273003905 +0000 2024-09-10T22:49:08.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:49:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-10T22:49:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-10T22:49:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000315845 s, 1.6 MB/s 2024-09-10T22:49:08.115 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:49:08.152 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-10T22:49:08.152 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:49:08.189 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:49:08.189 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-10T22:49:08.266 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:49:08.266 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:08.266 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 628 Links: 1 2024-09-10T22:49:08.266 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:08.267 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:08.267 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-10 22:48:44.466940329 +0000 2024-09-10T22:49:08.267 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-10 22:42:56.750031616 +0000 2024-09-10T22:49:08.267 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-10 22:42:56.750031616 +0000 2024-09-10T22:49:08.267 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-10 22:42:56.750031616 +0000 2024-09-10T22:49:08.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:49:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-10T22:49:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-10T22:49:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000288358 s, 1.8 MB/s 2024-09-10T22:49:08.363 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:49:08.401 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-10T22:49:08.477 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:49:08.477 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:08.477 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 645 Links: 1 2024-09-10T22:49:08.477 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:08.478 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:08.478 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-10 22:48:44.467940301 +0000 2024-09-10T22:49:08.478 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-10 22:42:57.249017351 +0000 2024-09-10T22:49:08.478 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-10 22:42:57.249017351 +0000 2024-09-10T22:49:08.478 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-10 22:42:57.249017351 +0000 2024-09-10T22:49:08.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:49:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-10T22:49:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-10T22:49:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000247397 s, 2.1 MB/s 2024-09-10T22:49:08.574 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:49:08.610 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-10T22:49:08.687 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:49:08.687 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:08.687 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 665 Links: 1 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-10 22:48:44.468940273 +0000 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-10 22:42:57.722003831 +0000 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-10 22:42:57.722003831 +0000 2024-09-10T22:49:08.688 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-10 22:42:57.722003831 +0000 2024-09-10T22:49:08.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:49:08.785 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-10T22:49:08.785 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-10T22:49:08.785 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000258855 s, 2.0 MB/s 2024-09-10T22:49:08.788 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:49:08.827 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 683 Links: 1 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-10 22:48:44.469940245 +0000 2024-09-10T22:49:08.905 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-10 22:42:58.155991424 +0000 2024-09-10T22:49:08.905 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-10 22:42:58.155991424 +0000 2024-09-10T22:49:08.905 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-10 22:42:58.155991424 +0000 2024-09-10T22:49:08.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:49:08.998 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-10T22:49:08.998 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-10T22:49:08.998 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000247348 s, 2.1 MB/s 2024-09-10T22:49:09.002 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:49:09.040 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T22:49:09.040 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:49:09.077 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:49:09.077 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-09-10T22:49:09.109 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.109 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.109 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:49:09.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:09.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:09.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:09.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:08 smithi145 ceph-mon[34397]: Reconfiguring daemon mon.c on smithi145 2024-09-10T22:49:09.148 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 22:48:41.500402813 +0000 2024-09-10T22:49:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 22:42:56.766293401 +0000 2024-09-10T22:49:09.150 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 22:42:56.766293401 +0000 2024-09-10T22:49:09.150 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 22:42:56.766293401 +0000 2024-09-10T22:49:09.150 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:49:09.240 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T22:49:09.240 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T22:49:09.240 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000266837 s, 1.9 MB/s 2024-09-10T22:49:09.243 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:09.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:08 smithi092 ceph-mon[34223]: Reconfiguring daemon mon.c on smithi145 2024-09-10T22:49:09.279 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-09-10T22:49:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:09.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:49:09.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:49:09.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:49:09.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:09.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:08 smithi067 ceph-mon[30939]: Reconfiguring daemon mon.c on smithi145 2024-09-10T22:49:09.358 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:49:09.358 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 22:48:41.500402813 +0000 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 22:42:57.255283099 +0000 2024-09-10T22:49:09.359 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 22:42:57.255283099 +0000 2024-09-10T22:49:09.360 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 22:42:57.255283099 +0000 2024-09-10T22:49:09.360 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:49:09.451 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T22:49:09.451 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T22:49:09.451 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000240603 s, 2.1 MB/s 2024-09-10T22:49:09.453 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:49:09.493 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-09-10T22:49:09.568 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 22:48:41.501402793 +0000 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 22:42:57.747272733 +0000 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 22:42:57.747272733 +0000 2024-09-10T22:49:09.569 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 22:42:57.747272733 +0000 2024-09-10T22:49:09.569 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:49:09.661 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T22:49:09.662 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T22:49:09.662 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000276469 s, 1.9 MB/s 2024-09-10T22:49:09.664 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:49:09.702 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-09-10T22:49:09.778 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 22:48:41.502402772 +0000 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 22:42:58.217262830 +0000 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 22:42:58.217262830 +0000 2024-09-10T22:49:09.779 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 22:42:58.217262830 +0000 2024-09-10T22:49:09.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:49:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T22:49:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T22:49:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000258882 s, 2.0 MB/s 2024-09-10T22:49:09.875 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:49:09.912 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-09-10T22:49:09.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:49:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:11.950 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:11 smithi067 ceph-mon[30939]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:11 smithi145 ceph-mon[34397]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:11 smithi092 ceph-mon[34223]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:13.160 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:49:13.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-09-10T22:49:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:13.957 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:13 smithi067 ceph-mon[30939]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 -- 172.21.15.67:0/4256596667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a41087b0 msgr2=0x7fe9a4108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/4256596667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a41087b0 0x7fe9a4108bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe990009900 tx=0x7fe99002a410 comp rx=0 tx=0).stop 2024-09-10T22:49:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 -- 172.21.15.67:0/4256596667 shutdown_connections 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/4256596667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a4109130 0x7fe9a410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/4256596667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a41087b0 0x7fe9a4108bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/4256596667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4102da0 0x7fe9a4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 -- 172.21.15.67:0/4256596667 >> 172.21.15.67:0/4256596667 conn(0x7fe9a40fe810 msgr2=0x7fe9a4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 -- 172.21.15.67:0/4256596667 shutdown_connections 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.072+0000 7fe9aaa00640 1 -- 172.21.15.67:0/4256596667 wait complete. 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 Processor -- start 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 -- start start 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a41087b0 0x7fe9a419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 0x7fe9a41a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4112c60 con 0x7fe9a4102da0 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4112ae0 con 0x7fe9a41087b0 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.073+0000 7fe9aaa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4112de0 con 0x7fe9a4109130 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60162/0 (socket says 172.21.15.67:60162) 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 -- 172.21.15.67:0/1479100259 learned_addr learned my addr 172.21.15.67:0/1479100259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3400640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a41087b0 0x7fe9a419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a8a00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 0x7fe9a41a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 -- 172.21.15.67:0/1479100259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 msgr2=0x7fe9a41a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 0x7fe9a41a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 -- 172.21.15.67:0/1479100259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a41087b0 msgr2=0x7fe9a419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a41087b0 0x7fe9a419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 -- 172.21.15.67:0/1479100259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a41a1460 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a8a00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 0x7fe9a41a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.074+0000 7fe9a3e00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d770 tx=0x7fe99400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.075+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994017070 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.075+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9940054b0 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.075+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a41a1750 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.075+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994004c60 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.075+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a7e10 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.076+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a4103180 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.078+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fe994004e00 con 0x7fe9a4102da0 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.079+0000 7fe9a0c00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 0x7fe9880b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.080+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe9940e3a70 con 0x7fe9a4102da0 2024-09-10T22:49:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.080+0000 7fe9a3400640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 0x7fe9880b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.081+0000 7fe9a3400640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 0x7fe9880b77e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9900057c0 tx=0x7fe9900058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.084+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe9940ac490 con 0x7fe9a4102da0 2024-09-10T22:49:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:13 smithi145 ceph-mon[34397]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:13 smithi092 ceph-mon[34223]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:14.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:14.290+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe9a4063590 con 0x7fe9880b5320 2024-09-10T22:49:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:15.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.235 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:15.888 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:15 smithi067 ceph-mon[30939]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:15.888 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:15 smithi067 ceph-mon[30939]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:15 smithi145 ceph-mon[34397]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:15 smithi145 ceph-mon[34397]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:15 smithi092 ceph-mon[34223]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:15 smithi092 ceph-mon[34223]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:17 smithi145 ceph-mon[34397]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"} : dispatch 2024-09-10T22:49:18.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"}]': finished 2024-09-10T22:49:18.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:17 smithi145 ceph-mon[34397]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:49:18.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:17 smithi092 ceph-mon[34223]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"} : dispatch 2024-09-10T22:49:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"}]': finished 2024-09-10T22:49:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:17 smithi092 ceph-mon[34223]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:49:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:17 smithi067 ceph-mon[30939]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"} : dispatch 2024-09-10T22:49:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4031741838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7bebf858-8d3f-46bb-aea6-043414353290"}]': finished 2024-09-10T22:49:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:17 smithi067 ceph-mon[30939]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:49:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:19.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1731708227' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1731708227' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:19.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1731708227' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:19 smithi092 ceph-mon[34223]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:19 smithi145 ceph-mon[34397]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:19 smithi067 ceph-mon[30939]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:20 smithi092 ceph-mon[34223]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:20 smithi067 ceph-mon[30939]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:20 smithi145 ceph-mon[34397]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:23.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:23 smithi067 ceph-mon[30939]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:23 smithi145 ceph-mon[34397]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:23.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:23 smithi092 ceph-mon[34223]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:25.272 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:25 smithi067 ceph-mon[30939]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:25 smithi145 ceph-mon[34397]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:25 smithi092 ceph-mon[34223]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:27.287 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:27 smithi067 ceph-mon[30939]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:27.287 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:49:27.287 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:27.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:27 smithi067 ceph-mon[30939]: Deploying daemon osd.0 on smithi067 2024-09-10T22:49:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:27 smithi145 ceph-mon[34397]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:49:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:27 smithi145 ceph-mon[34397]: Deploying daemon osd.0 on smithi067 2024-09-10T22:49:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:27 smithi092 ceph-mon[34223]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:49:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:27.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:27 smithi092 ceph-mon[34223]: Deploying daemon osd.0 on smithi067 2024-09-10T22:49:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:29 smithi067 ceph-mon[30939]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:29 smithi145 ceph-mon[34397]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:29.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:29 smithi092 ceph-mon[34223]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:31.260 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:31 smithi067 ceph-mon[30939]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:31 smithi145 ceph-mon[34397]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:31.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:31 smithi092 ceph-mon[34223]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:33.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:33 smithi067 ceph-mon[30939]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:33 smithi145 ceph-mon[34397]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:33.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:33 smithi092 ceph-mon[34223]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:35.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:35 smithi067 ceph-mon[30939]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:35.328 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:35.328 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:35.328 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:35.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:35 smithi145 ceph-mon[34397]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:35.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:35.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:35.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:35 smithi092 ceph-mon[34223]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:37.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:37 smithi067 ceph-mon[30939]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:37 smithi145 ceph-mon[34397]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:37 smithi092 ceph-mon[34223]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:39.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:39 smithi092 ceph-mon[34223]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:39 smithi067 ceph-mon[30939]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:39 smithi145 ceph-mon[34397]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:39.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.659+0000 7fe9a0c00640 1 -- 172.21.15.67:0/1479100259 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9a4063590 con 0x7fe9880b5320 2024-09-10T22:49:39.662 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 msgr2=0x7fe9880b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 0x7fe9880b77e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9900057c0 tx=0x7fe9900058d0 comp rx=0 tx=0).stop 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 msgr2=0x7fe9a419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d770 tx=0x7fe99400dc30 comp rx=0 tx=0).stop 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 shutdown_connections 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe9880b5320 0x7fe9880b77e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9a4109130 0x7fe9a41a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a41087b0 0x7fe9a419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 --2- 172.21.15.67:0/1479100259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4102da0 0x7fe9a419b3b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 >> 172.21.15.67:0/1479100259 conn(0x7fe9a40fe810 msgr2=0x7fe9a4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 shutdown_connections 2024-09-10T22:49:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:49:39.662+0000 7fe9aaa00640 1 -- 172.21.15.67:0/1479100259 wait complete. 2024-09-10T22:49:40.379 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:40.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.380 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.421 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T22:49:40.426 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-09-10T22:49:40.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:49:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:40.647 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:49:40 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T22:49:40.579+0000 7f1550961640 -1 osd.0 0 log_to_monitors true 2024-09-10T22:49:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:49:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:41 smithi067 ceph-mon[30939]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:41 smithi067 ceph-mon[30939]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:49:41.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:41 smithi092 ceph-mon[34223]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:41.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:41 smithi092 ceph-mon[34223]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:49:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:41 smithi145 ceph-mon[34397]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:41 smithi145 ceph-mon[34397]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:49:42.433 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:42 smithi092 ceph-mon[34223]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:49:42.433 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:42 smithi092 ceph-mon[34223]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:49:42.433 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:42 smithi092 ceph-mon[34223]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:49:42.434 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:42.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-mon[30939]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:49:42.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-mon[30939]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:49:42.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-mon[30939]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:49:42.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:42.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:42 smithi145 ceph-mon[34397]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:49:42.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:42 smithi145 ceph-mon[34397]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:49:42.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:42 smithi145 ceph-mon[34397]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:49:42.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.263 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T22:49:42.876+0000 7f154c200640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T22:49:43.263 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:49:42 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T22:49:42.892+0000 7f1541800640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:49:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:49:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T22:49:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:43 smithi092 ceph-mon[34223]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' 2024-09-10T22:49:43.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:43.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:49:43.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T22:49:43.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:43 smithi067 ceph-mon[30939]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' 2024-09-10T22:49:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:49:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T22:49:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:43 smithi145 ceph-mon[34397]: from='osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493]' entity='osd.0' 2024-09-10T22:49:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:49:43.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] boot 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:44.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:44.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:44.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] boot 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:44.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.601+0000 7fddd9a00640 1 -- 172.21.15.92:0/3829111719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd41045e0 msgr2=0x7fddd410cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.601+0000 7fddd9a00640 1 --2- 172.21.15.92:0/3829111719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd41045e0 0x7fddd410cad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009900 tx=0x7fddbc02a410 comp rx=0 tx=0).stop 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 -- 172.21.15.92:0/3829111719 shutdown_connections 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 --2- 172.21.15.92:0/3829111719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd410f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 --2- 172.21.15.92:0/3829111719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd41045e0 0x7fddd410cad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 --2- 172.21.15.92:0/3829111719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd4103cc0 0x7fddd41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 -- 172.21.15.92:0/3829111719 >> 172.21.15.92:0/3829111719 conn(0x7fddd40fd700 msgr2=0x7fddd40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 -- 172.21.15.92:0/3829111719 shutdown_connections 2024-09-10T22:49:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.602+0000 7fddd9a00640 1 -- 172.21.15.92:0/3829111719 wait complete. 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.603+0000 7fddd9a00640 1 Processor -- start 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.603+0000 7fddd9a00640 1 -- start start 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.603+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.603+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd41045e0 0x7fddd419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.603+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4112c40 con 0x7fddd410d120 2024-09-10T22:49:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd9a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4112ac0 con 0x7fddd4103cc0 2024-09-10T22:49:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4112dc0 con 0x7fddd41045e0 2024-09-10T22:49:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40190/0 (socket says 172.21.15.92:40190) 2024-09-10T22:49:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 -- 172.21.15.92:0/1764571640 learned_addr learned my addr 172.21.15.92:0/1764571640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd3400640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddcbe00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd41045e0 0x7fddd419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 -- 172.21.15.92:0/1764571640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd41045e0 msgr2=0x7fddd419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd41045e0 0x7fddd419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 -- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 msgr2=0x7fddd41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 -- 172.21.15.92:0/1764571640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd41a1470 con 0x7fddd4103cc0 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd3400640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.604+0000 7fddd2a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddc400c970 tx=0x7fddc400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.605+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4007bb0 con 0x7fddd4103cc0 2024-09-10T22:49:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.605+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddc4007d50 con 0x7fddd4103cc0 2024-09-10T22:49:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.605+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd41a1760 con 0x7fddd4103cc0 2024-09-10T22:49:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.605+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4004e40 con 0x7fddd4103cc0 2024-09-10T22:49:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.605+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd41a7e20 con 0x7fddd4103cc0 2024-09-10T22:49:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.606+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd41040a0 con 0x7fddd4103cc0 2024-09-10T22:49:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.608+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fddc4005020 con 0x7fddd4103cc0 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.610+0000 7fddcb400640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 0x7fddb80b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.611+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7fddc40e4070 con 0x7fddd4103cc0 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.611+0000 7fddcbe00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 0x7fddb80b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.611+0000 7fddcbe00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 0x7fddb80b77e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fddbc006010 tx=0x7fddbc032ce0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.615+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fddc4014030 con 0x7fddd4103cc0 2024-09-10T22:49:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:49:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:49:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: osd.0 [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] boot 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:49:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:49:44.809+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fddd4103080 con 0x7fddb80b5320 2024-09-10T22:49:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi067 2024-09-10T22:49:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 17089M 2024-09-10T22:49:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: from='client.34307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:45.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:45.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi067 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 17089M 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: from='client.34307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:49:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi067 2024-09-10T22:49:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 17089M 2024-09-10T22:49:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: from='client.34307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:46.362 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:46 smithi092 ceph-mon[34223]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T22:49:46.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:46 smithi067 ceph-mon[30939]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T22:49:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:46 smithi145 ceph-mon[34397]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T22:49:47.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:47 smithi092 ceph-mon[34223]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:47.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:47 smithi067 ceph-mon[30939]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:47 smithi145 ceph-mon[34397]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/3825333651' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:48 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:48 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"}]': finished 2024-09-10T22:49:48.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:48 smithi067 ceph-mon[30939]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:49:48.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:48 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:49:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/3825333651' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:48 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:48 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"}]': finished 2024-09-10T22:49:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:48 smithi145 ceph-mon[34397]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:49:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:48 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:49:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/3825333651' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:48 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"} : dispatch 2024-09-10T22:49:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:48 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "002adf57-737e-4e91-9cf5-164895bffa8e"}]': finished 2024-09-10T22:49:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:48 smithi092 ceph-mon[34223]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:49:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:48 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:49:49.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:49 smithi067 ceph-mon[30939]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:49.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/2274915575' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:49 smithi145 ceph-mon[34397]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/2274915575' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:49 smithi092 ceph-mon[34223]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/2274915575' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:51.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:51 smithi067 ceph-mon[30939]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:51 smithi145 ceph-mon[34397]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:51 smithi092 ceph-mon[34223]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:53 smithi067 ceph-mon[30939]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:53 smithi145 ceph-mon[34397]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:53 smithi092 ceph-mon[34223]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:55.353 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:55 smithi092 ceph-mon[34223]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:55 smithi067 ceph-mon[30939]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:55 smithi145 ceph-mon[34397]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:57 smithi092 ceph-mon[34223]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:49:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:57 smithi092 ceph-mon[34223]: Deploying daemon osd.1 on smithi092 2024-09-10T22:49:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:57 smithi067 ceph-mon[30939]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:49:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:57.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:57 smithi067 ceph-mon[30939]: Deploying daemon osd.1 on smithi092 2024-09-10T22:49:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:57 smithi145 ceph-mon[34397]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:49:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:57 smithi145 ceph-mon[34397]: Deploying daemon osd.1 on smithi092 2024-09-10T22:49:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:49:59 smithi145 ceph-mon[34397]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:59.665 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:49:59 smithi092 ceph-mon[34223]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:49:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:49:59 smithi067 ceph-mon[30939]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:00.497 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:00 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T22:50:00.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:00 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T22:50:00.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:00 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T22:50:01.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:01 smithi145 ceph-mon[34397]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:01 smithi092 ceph-mon[34223]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:01 smithi067 ceph-mon[30939]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:03.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:03 smithi067 ceph-mon[30939]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:03 smithi145 ceph-mon[34397]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:03.664 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:03 smithi092 ceph-mon[34223]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:04.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:04.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:04.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:04.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:04.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:04.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:05 smithi067 ceph-mon[30939]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:05.603 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:05 smithi092 ceph-mon[34223]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:05 smithi145 ceph-mon[34397]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:07.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:07.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:07.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:07.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:07.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:07.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:07.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.053+0000 7fddcb400640 1 -- 172.21.15.92:0/1764571640 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddd4103080 con 0x7fddb80b5320 2024-09-10T22:50:09.056 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 msgr2=0x7fddb80b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 0x7fddb80b77e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fddbc006010 tx=0x7fddbc032ce0 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 msgr2=0x7fddd419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddc400c970 tx=0x7fddc400ce30 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 shutdown_connections 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fddb80b5320 0x7fddb80b77e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd410d120 0x7fddd41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddd41045e0 0x7fddd419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 --2- 172.21.15.92:0/1764571640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd4103cc0 0x7fddd419b3c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 >> 172.21.15.92:0/1764571640 conn(0x7fddd40fd700 msgr2=0x7fddd40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 shutdown_connections 2024-09-10T22:50:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:50:09.056+0000 7fddd9a00640 1 -- 172.21.15.92:0/1764571640 wait complete. 2024-09-10T22:50:09.726 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:09 smithi092 ceph-mon[34223]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:09.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:09.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.727 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.764 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T22:50:09.769 INFO:tasks.cephadm:Deploying osd.2 on smithi145 with /dev/vg_nvme/lv_4... 2024-09-10T22:50:09.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:50:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:09 smithi067 ceph-mon[30939]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.823 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:09 smithi145 ceph-mon[34397]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:09.823 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:09.823 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:09.823 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:10.140 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:50:10.618 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:10 smithi145 ceph-mon[34397]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:10.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:10 smithi145 ceph-mon[34397]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:10.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:10 smithi092 ceph-mon[34223]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:10.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:10 smithi092 ceph-mon[34223]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:10.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:10 smithi067 ceph-mon[30939]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:10.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:10 smithi067 ceph-mon[30939]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:50:11.578 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:11.578 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:50:11.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:50:11.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:11.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:11.579 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-mon[34223]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:11.579 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1[39966]: 2024-09-10T22:50:11.508+0000 7f0f63a00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T22:50:11.579 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1[39966]: 2024-09-10T22:50:11.521+0000 7f0f5ea00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:50:11.687 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:11.687 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:50:11.687 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:50:11.687 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:11.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:11.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:11 smithi145 ceph-mon[34397]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:50:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:50:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:11.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: from='osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:11.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:11 smithi067 ceph-mon[30939]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:50:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:12 smithi092 ceph-mon[34223]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:50:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:12 smithi092 ceph-mon[34223]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T22:50:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:12 smithi067 ceph-mon[30939]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:50:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:12 smithi067 ceph-mon[30939]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T22:50:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:12 smithi145 ceph-mon[34397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:50:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:12 smithi145 ceph-mon[34397]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T22:50:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:13.294 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:50:13.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-09-10T22:50:13.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] boot 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:13.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.692 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] boot 2024-09-10T22:50:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:50:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:50:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:50:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: osd.1 [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] boot 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:14.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 -- 172.21.15.145:0/1310330834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810a9b0 msgr2=0x7f98e810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:14.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 --2- 172.21.15.145:0/1310330834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810a9b0 0x7f98e810adf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:14.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 -- 172.21.15.145:0/1310330834 shutdown_connections 2024-09-10T22:50:14.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 --2- 172.21.15.145:0/1310330834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810b330 0x7f98e8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:14.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 --2- 172.21.15.145:0/1310330834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810a9b0 0x7f98e810adf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 --2- 172.21.15.145:0/1310330834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 -- 172.21.15.145:0/1310330834 >> 172.21.15.145:0/1310330834 conn(0x7f98e8100a50 msgr2=0x7f98e8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 -- 172.21.15.145:0/1310330834 shutdown_connections 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.184+0000 7f98ed800640 1 -- 172.21.15.145:0/1310330834 wait complete. 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 Processor -- start 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 -- start start 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810a9b0 0x7f98e819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114de0 con 0x7f98e8104fa0 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114c60 con 0x7f98e810a9b0 2024-09-10T22:50:14.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98ed800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114f60 con 0x7f98e810b330 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:32902/0 (socket says 172.21.15.145:32902) 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.185+0000 7f98e7400640 1 -- 172.21.15.145:0/3066997692 learned_addr learned my addr 172.21.15.145:0/3066997692 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e6a00640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e6000640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810a9b0 0x7f98e819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 -- 172.21.15.145:0/3066997692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810a9b0 msgr2=0x7f98e819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810a9b0 0x7f98e819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 -- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 msgr2=0x7f98e819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 -- 172.21.15.145:0/3066997692 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e81a1420 con 0x7f98e810b330 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e6a00640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:14.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98e7400640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d750 tx=0x7f98e000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98e00054b0 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e81a1710 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.186+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004b90 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.187+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a7dd0 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.188+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e8105380 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.190+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f98e0004d30 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.191+0000 7f98df400640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 0x7f98cc0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.191+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2524+0+0 (secure 0 0 0) 0x7f98e00e3e70 con 0x7f98e810b330 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.191+0000 7f98e6a00640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 0x7f98cc0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.192+0000 7f98e6a00640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 0x7f98cc0b77e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005fd0 tx=0x7f98d8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:14.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.195+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f98e00abdc0 con 0x7f98e810b330 2024-09-10T22:50:14.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:14.391+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f98e8063590 con 0x7f98cc0b5320 2024-09-10T22:50:14.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi092 2024-09-10T22:50:14.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 17089M 2024-09-10T22:50:14.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T22:50:14.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:14.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:14.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi092 2024-09-10T22:50:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 17089M 2024-09-10T22:50:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T22:50:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:15.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi092 2024-09-10T22:50:15.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 17089M 2024-09-10T22:50:15.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T22:50:15.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:15.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:15.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:15 smithi145 ceph-mon[34397]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:15 smithi145 ceph-mon[34397]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:15 smithi067 ceph-mon[30939]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:15 smithi067 ceph-mon[30939]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:15.817 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:15 smithi092 ceph-mon[34223]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:15.817 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:15 smithi092 ceph-mon[34223]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/1735146552' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"}]': finished 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:50:17.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/1735146552' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"}]': finished 2024-09-10T22:50:17.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:50:17.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/1735146552' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"} : dispatch 2024-09-10T22:50:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210c2de7-0ad3-4e05-b098-446542372b6a"}]': finished 2024-09-10T22:50:18.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:50:18.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:18.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/2534492668' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:18.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/2534492668' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:19.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/2534492668' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:19 smithi067 ceph-mon[30939]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:19 smithi145 ceph-mon[34397]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:19 smithi092 ceph-mon[34223]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:21 smithi067 ceph-mon[30939]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:21 smithi145 ceph-mon[34397]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:21 smithi092 ceph-mon[34223]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:23 smithi067 ceph-mon[30939]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:23 smithi145 ceph-mon[34397]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:23 smithi092 ceph-mon[34223]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:25.825 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:25 smithi145 ceph-mon[34397]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:25.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:25 smithi067 ceph-mon[30939]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:25 smithi092 ceph-mon[34223]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:26.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:50:26.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:26.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:26 smithi145 ceph-mon[34397]: Deploying daemon osd.2 on smithi145 2024-09-10T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:27.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:26 smithi092 ceph-mon[34223]: Deploying daemon osd.2 on smithi145 2024-09-10T22:50:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:50:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:26 smithi067 ceph-mon[30939]: Deploying daemon osd.2 on smithi145 2024-09-10T22:50:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:27 smithi145 ceph-mon[34397]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:27 smithi092 ceph-mon[34223]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:27 smithi067 ceph-mon[30939]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:29.624 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:29 smithi145 ceph-mon[34397]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:29 smithi092 ceph-mon[34223]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:29 smithi067 ceph-mon[30939]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:31 smithi145 ceph-mon[34397]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:31 smithi092 ceph-mon[34223]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:31 smithi067 ceph-mon[30939]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:33.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:33 smithi145 ceph-mon[34397]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:33.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:33 smithi067 ceph-mon[30939]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:33 smithi092 ceph-mon[34223]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:34.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:34.709 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:34.709 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:35.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:35.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:35.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:35.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:35 smithi145 ceph-mon[34397]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:35 smithi092 ceph-mon[34223]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:35 smithi067 ceph-mon[30939]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:37.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:37.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:37.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:37.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:37.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:37.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.000 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 2 on host 'smithi145' 2024-09-10T22:50:39.000 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:38.998+0000 7f98df400640 1 -- 172.21.15.145:0/3066997692 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98e8063590 con 0x7f98cc0b5320 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 msgr2=0x7f98cc0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 0x7f98cc0b77e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005fd0 tx=0x7f98d8009500 comp rx=0 tx=0).stop 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 msgr2=0x7f98e81a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d750 tx=0x7f98e000dc10 comp rx=0 tx=0).stop 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 shutdown_connections 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98cc0b5320 0x7f98cc0b77e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98e810b330 0x7f98e81a0c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e810a9b0 0x7f98e819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:39.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 --2- 172.21.15.145:0/3066997692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98e8104fa0 0x7f98e819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:39.004 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.000+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 >> 172.21.15.145:0/3066997692 conn(0x7f98e8100a50 msgr2=0x7f98e8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:39.004 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.001+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 shutdown_connections 2024-09-10T22:50:39.004 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:50:39.001+0000 7f98ed800640 1 -- 172.21.15.145:0/3066997692 wait complete. 2024-09-10T22:50:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:39 smithi145 ceph-mon[34397]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.752 DEBUG:teuthology.orchestra.run.smithi145:osd.2> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T22:50:39.756 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2024-09-10T22:50:39.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:50:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:39 smithi092 ceph-mon[34223]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:39 smithi067 ceph-mon[30939]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:40.001 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T22:50:39.869+0000 7f94bd632640 -1 osd.2 0 log_to_monitors true 2024-09-10T22:50:40.122 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:40 smithi092 ceph-mon[34223]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:40 smithi092 ceph-mon[34223]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:40.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:40 smithi067 ceph-mon[30939]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:40.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:40 smithi067 ceph-mon[30939]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:40 smithi145 ceph-mon[34397]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:40 smithi145 ceph-mon[34397]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:50:41.630 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T22:50:41.437+0000 7f94b7e00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T22:50:41.631 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T22:50:41.469+0000 7f94b2e00640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:41.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:41 smithi145 ceph-mon[34397]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:41.675 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:41 smithi067 ceph-mon[30939]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:50:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: from='osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:50:41.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:41.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:41 smithi092 ceph-mon[34223]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:50:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:42 smithi092 ceph-mon[34223]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:50:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:42 smithi092 ceph-mon[34223]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T22:50:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:42.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:42 smithi067 ceph-mon[30939]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:50:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:42 smithi067 ceph-mon[30939]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T22:50:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:42 smithi145 ceph-mon[34397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:50:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:42 smithi145 ceph-mon[34397]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T22:50:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:43.180 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:50:43.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-09-10T22:50:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:50:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:50:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] boot 2024-09-10T22:50:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:50:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:43.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:43.633 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.646 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:50:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:50:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] boot 2024-09-10T22:50:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:43.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: osd.2 [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] boot 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:50:43.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:43.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 -- 172.21.15.67:0/3907988060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 msgr2=0x7f68e4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 --2- 172.21.15.67:0/3907988060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e4103180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009900 tx=0x7f68d802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 -- 172.21.15.67:0/3907988060 shutdown_connections 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 --2- 172.21.15.67:0/3907988060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 --2- 172.21.15.67:0/3907988060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 --2- 172.21.15.67:0/3907988060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e4103180 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 -- 172.21.15.67:0/3907988060 >> 172.21.15.67:0/3907988060 conn(0x7f68e40fe810 msgr2=0x7f68e4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 -- 172.21.15.67:0/3907988060 shutdown_connections 2024-09-10T22:50:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.209+0000 7f68ec200640 1 -- 172.21.15.67:0/3907988060 wait complete. 2024-09-10T22:50:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.210+0000 7f68ec200640 1 Processor -- start 2024-09-10T22:50:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.210+0000 7f68ec200640 1 -- start start 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.210+0000 7f68ec200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.210+0000 7f68ec200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ec200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ec200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112c60 con 0x7f68e4102da0 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ec200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112ae0 con 0x7f68e4109130 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ec200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112de0 con 0x7f68e41087b0 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44460/0 (socket says 172.21.15.67:44460) 2024-09-10T22:50:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 -- 172.21.15.67:0/4078037731 learned_addr learned my addr 172.21.15.67:0/4078037731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e8e00640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ea200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 -- 172.21.15.67:0/4078037731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 msgr2=0x7f68e419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 -- 172.21.15.67:0/4078037731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 msgr2=0x7f68e41a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e41a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e9800640 1 -- 172.21.15.67:0/4078037731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e41a14a0 con 0x7f68e4102da0 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68e8e00640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.211+0000 7f68ea200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68e9800640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009230 tx=0x7f68d80319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d803f030 con 0x7f68e4102da0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d80043d0 con 0x7f68e4102da0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8035070 con 0x7f68e4102da0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e41a1730 con 0x7f68e4102da0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.212+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41a7e50 con 0x7f68e4102da0 2024-09-10T22:50:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.213+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e4103180 con 0x7f68e4102da0 2024-09-10T22:50:44.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.215+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f68d803b880 con 0x7f68e4102da0 2024-09-10T22:50:44.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.217+0000 7f68e2000640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 0x7f68cc0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.217+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3066+0+0 (secure 0 0 0) 0x7f68d8105210 con 0x7f68e4102da0 2024-09-10T22:50:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.217+0000 7f68e8e00640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 0x7f68cc0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:44.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.218+0000 7f68e8e00640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 0x7f68cc0b77e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68e41082e0 tx=0x7f68d0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.219+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f68d80cd590 con 0x7f68e4102da0 2024-09-10T22:50:44.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:44.423+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f68e4063590 con 0x7f68cc0b5320 2024-09-10T22:50:44.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi145 2024-09-10T22:50:44.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 17089M 2024-09-10T22:50:44.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:50:44.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:44.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:44.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:44.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi145 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 17089M 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:44.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi145 2024-09-10T22:50:44.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 17089M 2024-09-10T22:50:44.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:50:44.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:50:44.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:50:44.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:44.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 ceph-mon[30939]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 ceph-mon[30939]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 ceph-mon[30939]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T22:50:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:45.721 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:45 smithi092 ceph-mon[34223]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:45.721 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:45 smithi092 ceph-mon[34223]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:45.721 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:45.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:45 smithi092 ceph-mon[34223]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T22:50:45.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:45.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:45.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:45.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43822]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:45.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43822]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:45.722 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:45.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 ceph-mon[34397]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:45.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 ceph-mon[34397]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:45.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:45.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 ceph-mon[34397]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T22:50:45.783 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:50:45.886 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53192]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53192]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53195]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53195]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:45.887 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53195]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.021 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.021 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43825]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.021 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 22:50:45 smithi092 sudo[43825]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.130 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:46.130 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.130 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44302]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.131 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44302]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.131 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:46.131 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.131 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44305]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.131 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44305]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44308]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44308]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:45 smithi145 sudo[44311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:46.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 sudo[44311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 sudo[44311]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 sudo[44311]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:46.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53199]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53199]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53202]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:46.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53202]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53202]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:45 smithi067 sudo[53202]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43828]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43828]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43828]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43828]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43831]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43831]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:50:46.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 sudo[43831]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:50:46.770 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:46 smithi067 ceph-mon[30939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.774 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:46 smithi092 ceph-mon[34223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:50:46.876+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f68d80c9100 con 0x7f68e4102da0 2024-09-10T22:50:46.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:50:46.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:46 smithi145 ceph-mon[34397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:50:47.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:47 smithi092 ceph-mon[34223]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:47.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:47 smithi092 ceph-mon[34223]: osdmap e23: 3 total, 3 up, 3 in 2024-09-10T22:50:47.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:47 smithi092 ceph-mon[34223]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T22:50:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:47 smithi067 ceph-mon[30939]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:47 smithi067 ceph-mon[30939]: osdmap e23: 3 total, 3 up, 3 in 2024-09-10T22:50:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:47 smithi067 ceph-mon[30939]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T22:50:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:47 smithi145 ceph-mon[34397]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:47 smithi145 ceph-mon[34397]: osdmap e23: 3 total, 3 up, 3 in 2024-09-10T22:50:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:47 smithi145 ceph-mon[34397]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T22:50:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"} : dispatch 2024-09-10T22:50:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"}]': finished 2024-09-10T22:50:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:48 smithi067 ceph-mon[30939]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T22:50:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:48 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:50:48.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/922245957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"} : dispatch 2024-09-10T22:50:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"}]': finished 2024-09-10T22:50:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:48 smithi092 ceph-mon[34223]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T22:50:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:48 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:50:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/922245957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:48.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"} : dispatch 2024-09-10T22:50:48.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1597752550' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b25a5d-e7a9-448a-b4ae-435368940e5d"}]': finished 2024-09-10T22:50:48.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:48 smithi145 ceph-mon[34397]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T22:50:48.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:48 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:50:48.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/922245957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:50:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:49 smithi145 ceph-mon[34397]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:50.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:49 smithi092 ceph-mon[34223]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:49 smithi067 ceph-mon[30939]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:51 smithi145 ceph-mon[34397]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:51 smithi092 ceph-mon[34223]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:51 smithi067 ceph-mon[30939]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:53 smithi145 ceph-mon[34397]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:53.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:53 smithi067 ceph-mon[30939]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:53 smithi092 ceph-mon[34223]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:55.643 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:55 smithi067 ceph-mon[30939]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:55 smithi145 ceph-mon[34397]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:55 smithi092 ceph-mon[34223]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:57.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:57 smithi067 ceph-mon[30939]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:57.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:50:57.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:57.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:57 smithi067 ceph-mon[30939]: Deploying daemon osd.3 on smithi067 2024-09-10T22:50:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:57 smithi145 ceph-mon[34397]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:50:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:57.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:57 smithi145 ceph-mon[34397]: Deploying daemon osd.3 on smithi067 2024-09-10T22:50:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:57 smithi092 ceph-mon[34223]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:50:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:58.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:57 smithi092 ceph-mon[34223]: Deploying daemon osd.3 on smithi067 2024-09-10T22:50:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:50:59 smithi067 ceph-mon[30939]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:50:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:50:59 smithi145 ceph-mon[34397]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:50:59 smithi092 ceph-mon[34223]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:01 smithi067 ceph-mon[30939]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:01 smithi145 ceph-mon[34397]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:01 smithi092 ceph-mon[34223]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:03.736 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:03 smithi067 ceph-mon[30939]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:03 smithi145 ceph-mon[34397]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:03 smithi092 ceph-mon[34223]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:04.858 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:04.859 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:04.859 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:05.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:05.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:05.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:05 smithi067 ceph-mon[30939]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:05 smithi145 ceph-mon[34397]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:05 smithi092 ceph-mon[34223]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:07 smithi067 ceph-mon[30939]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:07 smithi145 ceph-mon[34397]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:07 smithi092 ceph-mon[34223]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:09.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.145+0000 7f68e2000640 1 -- 172.21.15.67:0/4078037731 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68e4063590 con 0x7f68cc0b5320 2024-09-10T22:51:09.146 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.147+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 msgr2=0x7f68cc0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.147+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 0x7f68cc0b77e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68e41082e0 tx=0x7f68d0005f90 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.147+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 msgr2=0x7f68e419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.147+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009230 tx=0x7f68d80319a0 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.147+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 shutdown_connections 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68cc0b5320 0x7f68cc0b77e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e4109130 0x7f68e41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68e41087b0 0x7f68e419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 --2- 172.21.15.67:0/4078037731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68e4102da0 0x7f68e419b3f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 >> 172.21.15.67:0/4078037731 conn(0x7f68e40fe810 msgr2=0x7f68e4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 shutdown_connections 2024-09-10T22:51:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:51:09.148+0000 7f68ec200640 1 -- 172.21.15.67:0/4078037731 wait complete. 2024-09-10T22:51:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:09 smithi067 ceph-mon[30939]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:09.865 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T22:51:09.870 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-09-10T22:51:09.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:51:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:09 smithi092 ceph-mon[34223]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:09 smithi145 ceph-mon[34397]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:51:10.738 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 22:51:10 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T22:51:10.426+0000 7fda5411e640 -1 osd.3 0 log_to_monitors true 2024-09-10T22:51:11.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:10 smithi092 ceph-mon[34223]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:51:11.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:10 smithi067 ceph-mon[30939]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:51:11.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:10 smithi145 ceph-mon[34397]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:51:11.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:11 smithi092 ceph-mon[34223]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:11.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:11 smithi092 ceph-mon[34223]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:51:11.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:11 smithi092 ceph-mon[34223]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T22:51:11.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:11 smithi092 ceph-mon[34223]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:51:11.856 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:11.881 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:11 smithi067 ceph-mon[30939]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:11.881 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:11 smithi067 ceph-mon[30939]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:51:11.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:11 smithi067 ceph-mon[30939]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T22:51:11.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:11 smithi067 ceph-mon[30939]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:51:11.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:11 smithi145 ceph-mon[34397]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:11 smithi145 ceph-mon[34397]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:51:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:11 smithi145 ceph-mon[34397]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T22:51:12.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:11 smithi145 ceph-mon[34397]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:51:12.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:12.548 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T22:51:12.267+0000 7fda4e200640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T22:51:12.548 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T22:51:12.288+0000 7fda44200640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:51:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:12 smithi092 ceph-mon[34223]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:51:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:12 smithi092 ceph-mon[34223]: osdmap e26: 4 total, 3 up, 4 in 2024-09-10T22:51:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:12.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:12 smithi092 ceph-mon[34223]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' 2024-09-10T22:51:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-mon[30939]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:51:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-mon[30939]: osdmap e26: 4 total, 3 up, 4 in 2024-09-10T22:51:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:13.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:12 smithi067 ceph-mon[30939]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' 2024-09-10T22:51:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:12 smithi145 ceph-mon[34397]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:51:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:12 smithi145 ceph-mon[34397]: osdmap e26: 4 total, 3 up, 4 in 2024-09-10T22:51:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:12 smithi145 ceph-mon[34397]: from='osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074]' entity='osd.3' 2024-09-10T22:51:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:51:13.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-09-10T22:51:13.894 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:51:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] boot 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:13 smithi092 ceph-mon[34223]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] boot 2024-09-10T22:51:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T22:51:14.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:13 smithi067 ceph-mon[30939]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: osd.3 [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] boot 2024-09-10T22:51:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T22:51:14.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:51:14.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:13 smithi145 ceph-mon[34397]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T22:51:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.456+0000 7f1667e00640 1 -- 172.21.15.92:0/426733727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1660113990 msgr2=0x7f1660115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.456+0000 7f1667e00640 1 --2- 172.21.15.92:0/426733727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1660113990 0x7f1660115d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ca0 tx=0x7f165802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 -- 172.21.15.92:0/426733727 shutdown_connections 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 --2- 172.21.15.92:0/426733727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1660113990 0x7f1660115d80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 --2- 172.21.15.92:0/426733727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660077ff0 0x7f1660113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 --2- 172.21.15.92:0/426733727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16600776d0 0x7f1660077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 -- 172.21.15.92:0/426733727 >> 172.21.15.92:0/426733727 conn(0x7f1660100a50 msgr2=0x7f1660102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 -- 172.21.15.92:0/426733727 shutdown_connections 2024-09-10T22:51:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.457+0000 7f1667e00640 1 -- 172.21.15.92:0/426733727 wait complete. 2024-09-10T22:51:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 Processor -- start 2024-09-10T22:51:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 -- start start 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1660077ff0 0x7f166019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 0x7f16601a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1660119270 con 0x7f1660113990 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16601190f0 con 0x7f16600776d0 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1667e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16601193f0 con 0x7f1660077ff0 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1665400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1665400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54478/0 (socket says 172.21.15.92:54478) 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1665400640 1 -- 172.21.15.92:0/3753136601 learned_addr learned my addr 172.21.15.92:0/3753136601 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1664a00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1660077ff0 0x7f166019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.458+0000 7f1665e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 0x7f16601a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 -- 172.21.15.92:0/3753136601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1660077ff0 msgr2=0x7f166019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:14.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1660077ff0 0x7f166019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 -- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 msgr2=0x7f16601a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 0x7f16601a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 -- 172.21.15.92:0/3753136601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16601a5830 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 0x7f16601a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1665400640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f165400e9f0 tx=0x7f165400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165400cd80 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16540054b0 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601a5b20 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654010430 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.459+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601ac1e0 con 0x7f16600776d0 2024-09-10T22:51:14.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.461+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660077ab0 con 0x7f16600776d0 2024-09-10T22:51:14.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.463+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f1654005020 con 0x7f16600776d0 2024-09-10T22:51:14.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.464+0000 7f165e000640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 0x7f16480b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:14.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.464+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3884+0+0 (secure 0 0 0) 0x7f16540e4ff0 con 0x7f16600776d0 2024-09-10T22:51:14.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.464+0000 7f1664a00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 0x7f16480b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:14.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.465+0000 7f1664a00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 0x7f16480b77f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1660103860 tx=0x7f164c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:14.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.468+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f16540acf60 con 0x7f16600776d0 2024-09-10T22:51:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:14.665+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1660115a80 con 0x7f16480b5330 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: osdmap e29: 4 total, 4 up, 4 in 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi067 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.509 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:15.510 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: osdmap e29: 4 total, 4 up, 4 in 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi067 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:15.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:15.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: osdmap e29: 4 total, 4 up, 4 in 2024-09-10T22:51:15.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi067 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:17.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:17 smithi092 ceph-mon[34223]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:17.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:17 smithi067 ceph-mon[30939]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:17 smithi145 ceph-mon[34397]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:18.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/929321731' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:18 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:18 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"}]': finished 2024-09-10T22:51:18.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:18 smithi092 ceph-mon[34223]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T22:51:18.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/929321731' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:18 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:18 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"}]': finished 2024-09-10T22:51:18.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:18 smithi067 ceph-mon[30939]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T22:51:18.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:18.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/929321731' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:18 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"} : dispatch 2024-09-10T22:51:18.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:18 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c7f079-20a7-4726-bf63-797bd66fc512"}]': finished 2024-09-10T22:51:18.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:18 smithi145 ceph-mon[34397]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T22:51:18.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:19.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:19 smithi092 ceph-mon[34223]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:19.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/1262951618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:19.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:19 smithi067 ceph-mon[30939]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:19.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/1262951618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:19 smithi145 ceph-mon[34397]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/1262951618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:21.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:21 smithi092 ceph-mon[34223]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:51:21.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:21 smithi067 ceph-mon[30939]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:51:21.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:21 smithi145 ceph-mon[34397]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:51:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:23 smithi092 ceph-mon[34223]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:51:23.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:23 smithi067 ceph-mon[30939]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:51:23.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:23 smithi145 ceph-mon[34397]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:51:25.449 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:25 smithi092 ceph-mon[34223]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:51:25.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:25 smithi067 ceph-mon[30939]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:51:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:25 smithi145 ceph-mon[34397]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:51:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:27 smithi092 ceph-mon[34223]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:51:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:51:27.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:27.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:27 smithi092 ceph-mon[34223]: Deploying daemon osd.4 on smithi092 2024-09-10T22:51:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:27 smithi067 ceph-mon[30939]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:51:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:51:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:27 smithi067 ceph-mon[30939]: Deploying daemon osd.4 on smithi092 2024-09-10T22:51:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:27 smithi145 ceph-mon[34397]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:51:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:51:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:27 smithi145 ceph-mon[34397]: Deploying daemon osd.4 on smithi092 2024-09-10T22:51:29.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:29 smithi092 ceph-mon[34223]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:51:29.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:29 smithi067 ceph-mon[30939]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:51:29.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:29 smithi145 ceph-mon[34397]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:51:31.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:31 smithi092 ceph-mon[34223]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:51:31.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:31 smithi067 ceph-mon[30939]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:51:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:31 smithi145 ceph-mon[34397]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:51:33.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:33 smithi092 ceph-mon[34223]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:33 smithi067 ceph-mon[30939]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:33 smithi145 ceph-mon[34397]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:34.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:34.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:34.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:35.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:35 smithi067 ceph-mon[30939]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:35 smithi092 ceph-mon[34223]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:35.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:35 smithi145 ceph-mon[34397]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:37 smithi092 ceph-mon[34223]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:37 smithi067 ceph-mon[30939]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:37 smithi145 ceph-mon[34397]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:38.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:38.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:38.748 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-09-10T22:51:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.747+0000 7f165e000640 1 -- 172.21.15.92:0/3753136601 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1660115a80 con 0x7f16480b5330 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 msgr2=0x7f16480b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 0x7f16480b77f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1660103860 tx=0x7f164c005f90 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 msgr2=0x7f166019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f165400e9f0 tx=0x7f165400edf0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 shutdown_connections 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f16480b5330 0x7f16480b77f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660113990 0x7f16601a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1660077ff0 0x7f166019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 --2- 172.21.15.92:0/3753136601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16600776d0 0x7f166019f780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 >> 172.21.15.92:0/3753136601 conn(0x7f1660100a50 msgr2=0x7f1660102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 shutdown_connections 2024-09-10T22:51:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:51:38.749+0000 7f1667e00640 1 -- 172.21.15.92:0/3753136601 wait complete. 2024-09-10T22:51:39.478 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T22:51:39.483 INFO:tasks.cephadm:Deploying osd.5 on smithi145 with /dev/vg_nvme/lv_3... 2024-09-10T22:51:39.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:51:39.530 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:39 smithi092 ceph-mon[34223]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:39.530 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:39.530 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.530 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:39 smithi145 ceph-mon[34397]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:39.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:39.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:39 smithi067 ceph-mon[30939]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:39.854 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:51:40.687 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 22:51:40 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T22:51:40.558+0000 7fa3c2297640 -1 osd.4 0 log_to_monitors true 2024-09-10T22:51:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:41 smithi092 ceph-mon[34223]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:41 smithi092 ceph-mon[34223]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:41 smithi092 ceph-mon[34223]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:41.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:41 smithi067 ceph-mon[30939]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:41.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:41 smithi067 ceph-mon[30939]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:41.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:41 smithi067 ceph-mon[30939]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:41.574 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:41 smithi145 ceph-mon[34397]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:41.575 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:41 smithi145 ceph-mon[34397]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:41.575 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:41 smithi145 ceph-mon[34397]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:51:42.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:42 smithi145 ceph-mon[34397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:51:42.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:42 smithi145 ceph-mon[34397]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:42.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:42 smithi145 ceph-mon[34397]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T22:51:42.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:42.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:42 smithi145 ceph-mon[34397]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:42.650 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T22:51:42.306+0000 7fa3bd200640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T22:51:42.651 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T22:51:42.316+0000 7fa3b8200640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:51:42.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-mon[34223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:51:42.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-mon[34223]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:42.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-mon[34223]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T22:51:42.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:42.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:42 smithi092 ceph-mon[34223]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:42 smithi067 ceph-mon[30939]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:51:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:42 smithi067 ceph-mon[30939]: from='osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:42 smithi067 ceph-mon[30939]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T22:51:42.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:42.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:42 smithi067 ceph-mon[30939]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:51:43.171 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:51:43.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-09-10T22:51:43.571 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:51:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:43.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:51:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:43.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:43.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:51:44.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 -- 172.21.15.145:0/812005388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 msgr2=0x7f3028108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 --2- 172.21.15.145:0/812005388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f3028108a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3010009900 tx=0x7f301002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 -- 172.21.15.145:0/812005388 shutdown_connections 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 --2- 172.21.15.145:0/812005388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028108fa0 0x7f302810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 --2- 172.21.15.145:0/812005388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f3028108a60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 --2- 172.21.15.145:0/812005388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028102c10 0x7f3028102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 -- 172.21.15.145:0/812005388 >> 172.21.15.145:0/812005388 conn(0x7f30280fe6c0 msgr2=0x7f3028100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 -- 172.21.15.145:0/812005388 shutdown_connections 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.119+0000 7f302cc00640 1 -- 172.21.15.145:0/812005388 wait complete. 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.120+0000 7f302cc00640 1 Processor -- start 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 -- start start 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028102c10 0x7f302819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028108fa0 0x7f30281a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3028112b20 con 0x7f3028108fa0 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30281129a0 con 0x7f3028102c10 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f302cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3028112ca0 con 0x7f3028108620 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f301ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f301ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51254/0 (socket says 172.21.15.145:51254) 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f301ea00640 1 -- 172.21.15.145:0/671372806 learned_addr learned my addr 172.21.15.145:0/671372806 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.121+0000 7f3027400640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028102c10 0x7f302819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f3027e00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028108fa0 0x7f30281a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 -- 172.21.15.145:0/671372806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028102c10 msgr2=0x7f302819b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028102c10 0x7f302819b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 -- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028108fa0 msgr2=0x7f30281a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028108fa0 0x7f30281a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 -- 172.21.15.145:0/671372806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30281a11b0 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f301ea00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f301002a910 tx=0x7f3010003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301003f070 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30281a1440 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3010003e40 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010035030 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.122+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281a7b60 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.123+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3028102ff0 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.125+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f3010031b90 con 0x7f3028108620 2024-09-10T22:51:44.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.127+0000 7f3024c00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 0x7f300c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.127+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4316+0+0 (secure 0 0 0) 0x7f3010105b90 con 0x7f3028108620 2024-09-10T22:51:44.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.127+0000 7f3027400640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 0x7f300c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.128+0000 7f3027400640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 0x7f300c0b78a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f301800abe0 tx=0x7f301800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:44.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.131+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f30100cd950 con 0x7f3028108620 2024-09-10T22:51:44.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:51:44.331+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3028063590 con 0x7f300c0b53e0 2024-09-10T22:51:44.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi092 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] boot 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T22:51:44.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi092 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] boot 2024-09-10T22:51:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T22:51:44.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:51:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi092 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: osd.4 [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] boot 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T22:51:44.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:51:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T22:51:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:45.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T22:51:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:45.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:45.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:51:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T22:51:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:51:45.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:51:45.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:47 smithi145 ceph-mon[34397]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:47.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:47 smithi092 ceph-mon[34223]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:47 smithi067 ceph-mon[30939]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/3681544605' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"}]': finished 2024-09-10T22:51:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T22:51:48.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:51:48.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/841703277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/3681544605' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"}]': finished 2024-09-10T22:51:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T22:51:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:51:48.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/841703277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:48.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/3681544605' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"} : dispatch 2024-09-10T22:51:48.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c623a06-702d-41dc-a7a7-56bc739d3199"}]': finished 2024-09-10T22:51:48.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T22:51:48.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:51:48.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/841703277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:51:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:49 smithi092 ceph-mon[34223]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:49 smithi067 ceph-mon[30939]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:49 smithi145 ceph-mon[34397]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:51 smithi092 ceph-mon[34223]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:51.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:51 smithi067 ceph-mon[30939]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:51.861 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:51 smithi145 ceph-mon[34397]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:53 smithi092 ceph-mon[34223]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:53 smithi067 ceph-mon[30939]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:53.807 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:53 smithi145 ceph-mon[34397]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:55.624 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:55 smithi145 ceph-mon[34397]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:55 smithi092 ceph-mon[34223]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:55 smithi067 ceph-mon[30939]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:56.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:51:56.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:56.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:51:56.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:56.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:51:56.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:57 smithi145 ceph-mon[34397]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:57 smithi145 ceph-mon[34397]: Deploying daemon osd.5 on smithi145 2024-09-10T22:51:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:57 smithi092 ceph-mon[34223]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:57 smithi092 ceph-mon[34223]: Deploying daemon osd.5 on smithi145 2024-09-10T22:51:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:57 smithi067 ceph-mon[30939]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:57 smithi067 ceph-mon[30939]: Deploying daemon osd.5 on smithi145 2024-09-10T22:51:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:51:59 smithi092 ceph-mon[34223]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:51:59 smithi067 ceph-mon[30939]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:51:59.799 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:51:59 smithi145 ceph-mon[34397]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:01 smithi092 ceph-mon[34223]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:01 smithi067 ceph-mon[30939]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:01 smithi145 ceph-mon[34397]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:03 smithi092 ceph-mon[34223]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:03.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:03 smithi067 ceph-mon[30939]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:03 smithi145 ceph-mon[34397]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:04.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:04.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:04.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:04.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:04.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:04.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:04.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:04.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:04.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:05.629 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:05 smithi145 ceph-mon[34397]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:05 smithi092 ceph-mon[34223]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:05 smithi067 ceph-mon[30939]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:07 smithi067 ceph-mon[30939]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:07 smithi145 ceph-mon[34397]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:07 smithi092 ceph-mon[34223]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:09.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:09.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.096+0000 7f3024c00640 1 -- 172.21.15.145:0/671372806 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3028063590 con 0x7f300c0b53e0 2024-09-10T22:52:09.098 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 5 on host 'smithi145' 2024-09-10T22:52:09.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 msgr2=0x7f300c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 0x7f300c0b78a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f301800abe0 tx=0x7f301800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 msgr2=0x7f302819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f301002a910 tx=0x7f3010003c20 comp rx=0 tx=0).stop 2024-09-10T22:52:09.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 shutdown_connections 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f300c0b53e0 0x7f300c0b78a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3028108fa0 0x7f30281a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3028108620 0x7f302819b640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 --2- 172.21.15.145:0/671372806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3028102c10 0x7f302819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 >> 172.21.15.145:0/671372806 conn(0x7f30280fe6c0 msgr2=0x7f3028101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 shutdown_connections 2024-09-10T22:52:09.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:52:09.099+0000 7f302cc00640 1 -- 172.21.15.145:0/671372806 wait complete. 2024-09-10T22:52:09.832 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:09 smithi145 ceph-mon[34397]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:09.833 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:09.833 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.833 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.868 DEBUG:teuthology.orchestra.run.smithi145:osd.5> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T22:52:09.873 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-09-10T22:52:09.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:52:09.922 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:09 smithi092 ceph-mon[34223]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:09.923 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:09.923 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.923 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:09 smithi067 ceph-mon[30939]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:09.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:09.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:09.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:10.251 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:10.361 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 22:52:10 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T22:52:10.116+0000 7f1765645640 -1 osd.5 0 log_to_monitors true 2024-09-10T22:52:10.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:10 smithi145 ceph-mon[34397]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:10.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:10 smithi145 ceph-mon[34397]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:10.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:10 smithi067 ceph-mon[30939]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:10.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:10 smithi067 ceph-mon[30939]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:11.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:10 smithi092 ceph-mon[34223]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:11.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:10 smithi092 ceph-mon[34223]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:52:11.785 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T22:52:11.601+0000 7f175fe00640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T22:52:11.785 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T22:52:11.609+0000 7f175ae00640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:52:11.785 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:11.785 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:52:11.785 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T22:52:11.785 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:11.786 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:11.786 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:11 smithi145 ceph-mon[34397]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:11.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:11 smithi067 ceph-mon[30939]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:52:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T22:52:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:12.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: from='osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:12.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:11 smithi092 ceph-mon[34223]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:52:12.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:12 smithi067 ceph-mon[30939]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:52:12.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:12 smithi067 ceph-mon[30939]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T22:52:12.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:12.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:12 smithi145 ceph-mon[34397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:52:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:12 smithi145 ceph-mon[34397]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T22:52:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:12 smithi092 ceph-mon[34223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:52:13.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:12 smithi092 ceph-mon[34223]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T22:52:13.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:52:13.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-09-10T22:52:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:52:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:52:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] boot 2024-09-10T22:52:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T22:52:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:13 smithi067 ceph-mon[30939]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] boot 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:13 smithi145 ceph-mon[34397]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T22:52:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:52:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:52:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:52:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: osd.5 [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] boot 2024-09-10T22:52:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T22:52:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:52:14.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:13 smithi092 ceph-mon[34223]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T22:52:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 -- 172.21.15.67:0/3388459119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024102dc0 msgr2=0x7f30241031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 --2- 172.21.15.67:0/3388459119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024102dc0 0x7f30241031a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3014009900 tx=0x7f301402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 -- 172.21.15.67:0/3388459119 shutdown_connections 2024-09-10T22:52:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 --2- 172.21.15.67:0/3388459119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3024109150 0x7f302410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 --2- 172.21.15.67:0/3388459119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30241087d0 0x7f3024108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 --2- 172.21.15.67:0/3388459119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024102dc0 0x7f30241031a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 -- 172.21.15.67:0/3388459119 >> 172.21.15.67:0/3388459119 conn(0x7f30240fe830 msgr2=0x7f3024100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 -- 172.21.15.67:0/3388459119 shutdown_connections 2024-09-10T22:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.529+0000 7f302ac00640 1 -- 172.21.15.67:0/3388459119 wait complete. 2024-09-10T22:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 Processor -- start 2024-09-10T22:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 -- start start 2024-09-10T22:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 0x7f302419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30241087d0 0x7f302419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3024112d10 con 0x7f3024109150 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3024112b90 con 0x7f30241087d0 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.530+0000 7f302ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3024112e90 con 0x7f3024102dc0 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3028c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 0x7f302419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3028c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39272/0 (socket says 172.21.15.67:39272) 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3028c00640 1 -- 172.21.15.67:0/3963144128 learned_addr learned my addr 172.21.15.67:0/3963144128 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f301b400640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30241087d0 0x7f302419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 -- 172.21.15.67:0/3963144128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30241087d0 msgr2=0x7f302419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30241087d0 0x7f302419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 -- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 msgr2=0x7f30241a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 -- 172.21.15.67:0/3963144128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30241a14a0 con 0x7f3024102dc0 2024-09-10T22:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3028c00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.531+0000 7f3023e00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 0x7f302419b3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f301402a910 tx=0x7f3014003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.532+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301403f070 con 0x7f3024102dc0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.532+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3014003e10 con 0x7f3024102dc0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.532+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014035030 con 0x7f3024102dc0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.532+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241a1730 con 0x7f3024102dc0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.532+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241a7e50 con 0x7f3024102dc0 2024-09-10T22:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.533+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30241031a0 con 0x7f3024102dc0 2024-09-10T22:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.535+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f3014031400 con 0x7f3024102dc0 2024-09-10T22:52:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.537+0000 7f3021600640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 0x7f30080b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.537+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4748+0+0 (secure 0 0 0) 0x7f3014105dd0 con 0x7f3024102dc0 2024-09-10T22:52:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.537+0000 7f301b400640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 0x7f30080b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.537+0000 7f301b400640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 0x7f30080b7870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30240ffb50 tx=0x7f300c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.538+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f30140cd390 con 0x7f3024102dc0 2024-09-10T22:52:14.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:14.746+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3024106fb0 con 0x7f30080b53b0 2024-09-10T22:52:15.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi145 2024-09-10T22:52:15.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:15.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi145 2024-09-10T22:52:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi145 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:15.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T22:52:15.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:15.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:15.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:15 smithi067 ceph-mon[30939]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:15 smithi067 ceph-mon[30939]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:15 smithi092 ceph-mon[34223]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:15 smithi092 ceph-mon[34223]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:15 smithi145 ceph-mon[34397]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:15 smithi145 ceph-mon[34397]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:17 smithi092 ceph-mon[34223]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:17 smithi067 ceph-mon[30939]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:17 smithi145 ceph-mon[34397]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:19.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"} : dispatch 2024-09-10T22:52:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"}]': finished 2024-09-10T22:52:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:18 smithi067 ceph-mon[30939]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T22:52:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:19.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/753620266' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"} : dispatch 2024-09-10T22:52:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"}]': finished 2024-09-10T22:52:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:18 smithi092 ceph-mon[34223]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T22:52:19.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:19.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/753620266' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"} : dispatch 2024-09-10T22:52:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/949150308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ca47089-cde1-479f-95ed-70c93dfafcf8"}]': finished 2024-09-10T22:52:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:18 smithi145 ceph-mon[34397]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T22:52:19.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:19.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/753620266' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:19 smithi092 ceph-mon[34223]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:19 smithi067 ceph-mon[30939]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:19 smithi145 ceph-mon[34397]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:21 smithi092 ceph-mon[34223]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:52:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:21 smithi067 ceph-mon[30939]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:52:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:21 smithi145 ceph-mon[34397]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:52:23.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:22 smithi067 ceph-mon[30939]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:52:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:22 smithi092 ceph-mon[34223]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:52:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:22 smithi145 ceph-mon[34397]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:52:25.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:25 smithi067 ceph-mon[30939]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:52:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:25 smithi145 ceph-mon[34397]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:52:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:25 smithi092 ceph-mon[34223]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:52:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:27 smithi145 ceph-mon[34397]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:52:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:27 smithi092 ceph-mon[34223]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:52:27.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:27 smithi067 ceph-mon[30939]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:52:28.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:28 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:52:28.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:28 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:28.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:28 smithi067 ceph-mon[30939]: Deploying daemon osd.6 on smithi067 2024-09-10T22:52:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:28 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:52:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:28 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:28 smithi145 ceph-mon[34397]: Deploying daemon osd.6 on smithi067 2024-09-10T22:52:28.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:28 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:52:28.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:28 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:28.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:28 smithi092 ceph-mon[34223]: Deploying daemon osd.6 on smithi067 2024-09-10T22:52:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:29 smithi067 ceph-mon[30939]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T22:52:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:29 smithi145 ceph-mon[34397]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T22:52:29.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:29 smithi092 ceph-mon[34223]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T22:52:31.216 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:31 smithi067 ceph-mon[30939]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:52:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:31 smithi145 ceph-mon[34397]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:52:31.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:31 smithi092 ceph-mon[34223]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:52:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:33 smithi145 ceph-mon[34397]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:33.450 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:33 smithi067 ceph-mon[30939]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:33.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:33 smithi092 ceph-mon[34223]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:35.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:35 smithi067 ceph-mon[30939]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:35.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:35 smithi145 ceph-mon[34397]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:35 smithi092 ceph-mon[34223]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:36.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:36.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:36.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:37.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:37 smithi067 ceph-mon[30939]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:37 smithi092 ceph-mon[34223]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:37 smithi145 ceph-mon[34397]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:39.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:39.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:39.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:39.894 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-09-10T22:52:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.893+0000 7f3021600640 1 -- 172.21.15.67:0/3963144128 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3024106fb0 con 0x7f30080b53b0 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 msgr2=0x7f30080b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 0x7f30080b7870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30240ffb50 tx=0x7f300c00b410 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 msgr2=0x7f302419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 0x7f302419b3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f301402a910 tx=0x7f3014003c20 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 shutdown_connections 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f30080b53b0 0x7f30080b7870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024109150 0x7f30241a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30241087d0 0x7f302419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 --2- 172.21.15.67:0/3963144128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3024102dc0 0x7f302419b3f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.895+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 >> 172.21.15.67:0/3963144128 conn(0x7f30240fe830 msgr2=0x7f30241068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.896+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 shutdown_connections 2024-09-10T22:52:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:52:39.896+0000 7f302ac00640 1 -- 172.21.15.67:0/3963144128 wait complete. 2024-09-10T22:52:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:40.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:40.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:40.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:40.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.771 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T22:52:40.775 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-09-10T22:52:40.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:52:40.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:40.999 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:40.999 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:41.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:41.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:41.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:52:41.548 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 22:52:41 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T22:52:41.399+0000 7f2a8602f640 -1 osd.6 0 log_to_monitors true 2024-09-10T22:52:42.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:41 smithi067 ceph-mon[30939]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:42.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:41 smithi067 ceph-mon[30939]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:52:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:41 smithi092 ceph-mon[34223]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:41 smithi092 ceph-mon[34223]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:52:42.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:41 smithi145 ceph-mon[34397]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:42.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:41 smithi145 ceph-mon[34397]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:52:42.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:42 smithi092 ceph-mon[34223]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:52:42.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:42 smithi092 ceph-mon[34223]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T22:52:42.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:42 smithi092 ceph-mon[34223]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:52:42.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:42.978 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:42 smithi092 ceph-mon[34223]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:52:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-mon[30939]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:52:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-mon[30939]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T22:52:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-mon[30939]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:52:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:43.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-mon[30939]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:52:43.299 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T22:52:42.913+0000 7f2a80800640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T22:52:43.299 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 22:52:42 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T22:52:42.925+0000 7f2a7b800640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:52:43.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:42 smithi145 ceph-mon[34397]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:52:43.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:42 smithi145 ceph-mon[34397]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T22:52:43.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:42 smithi145 ceph-mon[34397]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:52:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:42 smithi145 ceph-mon[34397]: from='osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:52:44.250 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:43 smithi092 ceph-mon[34223]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:44.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:43 smithi092 ceph-mon[34223]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T22:52:44.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.251 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:43 smithi067 ceph-mon[30939]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:43 smithi067 ceph-mon[30939]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T22:52:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:43 smithi145 ceph-mon[34397]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:43 smithi145 ceph-mon[34397]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T22:52:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:52:44.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-09-10T22:52:45.009 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:52:45.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:52:45.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] boot 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] boot 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.159 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.160 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.160 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:45.160 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:45.160 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:52:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: osd.6 [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] boot 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:52:45.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.541+0000 7ff5e8200640 1 -- 172.21.15.92:0/1218222615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 msgr2=0x7ff5e010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.541+0000 7ff5e8200640 1 --2- 172.21.15.92:0/1218222615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e010adf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009900 tx=0x7ff5cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 -- 172.21.15.92:0/1218222615 shutdown_connections 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 --2- 172.21.15.92:0/1218222615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 0x7ff5e0111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 --2- 172.21.15.92:0/1218222615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e010adf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 --2- 172.21.15.92:0/1218222615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 -- 172.21.15.92:0/1218222615 >> 172.21.15.92:0/1218222615 conn(0x7ff5e0100a50 msgr2=0x7ff5e0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 -- 172.21.15.92:0/1218222615 shutdown_connections 2024-09-10T22:52:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 -- 172.21.15.92:0/1218222615 wait complete. 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.542+0000 7ff5e8200640 1 Processor -- start 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 -- start start 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 0x7ff5e01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114e30 con 0x7ff5e0104fa0 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114cb0 con 0x7ff5e010a9b0 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114fb0 con 0x7ff5e010b330 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33104/0 (socket says 172.21.15.92:33104) 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.543+0000 7ff5e4e00640 1 -- 172.21.15.92:0/2448720 learned_addr learned my addr 172.21.15.92:0/2448720 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e6200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 0x7ff5e01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 -- 172.21.15.92:0/2448720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 msgr2=0x7ff5e01a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 0x7ff5e01a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 -- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 msgr2=0x7ff5e019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 -- 172.21.15.92:0/2448720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e01a13f0 con 0x7ff5e010a9b0 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e5800640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.544+0000 7ff5e4e00640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc02a910 tx=0x7ff5cc003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.545+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc03f070 con 0x7ff5e010a9b0 2024-09-10T22:52:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.545+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5cc003e40 con 0x7ff5e010a9b0 2024-09-10T22:52:45.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.545+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc035030 con 0x7ff5e010a9b0 2024-09-10T22:52:45.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.545+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e01a1680 con 0x7ff5e010a9b0 2024-09-10T22:52:45.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.545+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01a7da0 con 0x7ff5e010a9b0 2024-09-10T22:52:45.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.546+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e0105380 con 0x7ff5e010a9b0 2024-09-10T22:52:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.548+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7ff5cc031400 con 0x7ff5e010a9b0 2024-09-10T22:52:45.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.549+0000 7ff5de000640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 0x7ff5c80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:45.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.550+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5180+0+0 (secure 0 0 0) 0x7ff5cc105e00 con 0x7ff5e010a9b0 2024-09-10T22:52:45.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.550+0000 7ff5e5800640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 0x7ff5c80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.551+0000 7ff5e5800640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 0x7ff5c80b7870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400f650 tx=0x7ff5d40075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.551+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff5cc0cd210 con 0x7ff5e010a9b0 2024-09-10T22:52:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:52:45.742+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff5e0063590 con 0x7ff5c80b53b0 2024-09-10T22:52:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi067 2024-09-10T22:52:46.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-10T22:52:46.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T22:52:46.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:46.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:46.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi067 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:52:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi067 2024-09-10T22:52:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-10T22:52:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T22:52:46.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:52:46.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:52:46.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:47.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:46 smithi092 ceph-mon[34223]: from='client.34463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:47.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:46 smithi067 ceph-mon[30939]: from='client.34463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:47.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:46 smithi145 ceph-mon[34397]: from='client.34463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:52:48.188 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:47 smithi092 ceph-mon[34223]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:47 smithi067 ceph-mon[30939]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:47 smithi145 ceph-mon[34397]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:48 smithi092 ceph-mon[34223]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:49.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:48 smithi067 ceph-mon[30939]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:49.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:48 smithi145 ceph-mon[34397]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/4181094888' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"}]': finished 2024-09-10T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:52:50.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/4227257621' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/4181094888' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"}]': finished 2024-09-10T22:52:50.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T22:52:50.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:52:50.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/4227257621' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/4181094888' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"} : dispatch 2024-09-10T22:52:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "726d7795-37f9-452e-a9a0-d764920275a2"}]': finished 2024-09-10T22:52:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T22:52:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:52:50.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/4227257621' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:52:51.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:50 smithi092 ceph-mon[34223]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:51.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:50 smithi067 ceph-mon[30939]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:51.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:50 smithi145 ceph-mon[34397]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:53.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:53 smithi145 ceph-mon[34397]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:53.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:53 smithi092 ceph-mon[34223]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:53 smithi067 ceph-mon[30939]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:55.211 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:55 smithi092 ceph-mon[34223]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:55.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:55 smithi145 ceph-mon[34397]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:55 smithi067 ceph-mon[30939]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:57.323 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:57 smithi092 ceph-mon[34223]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:57.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:57 smithi145 ceph-mon[34397]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:57 smithi067 ceph-mon[30939]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:58 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:52:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:58 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:58 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:52:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:58 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:58.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:58 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:52:58.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:58 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:59.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:59 smithi092 ceph-mon[34223]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:59.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:52:59 smithi092 ceph-mon[34223]: Deploying daemon osd.7 on smithi092 2024-09-10T22:52:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:59 smithi145 ceph-mon[34397]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:52:59 smithi145 ceph-mon[34397]: Deploying daemon osd.7 on smithi092 2024-09-10T22:52:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:59 smithi067 ceph-mon[30939]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:52:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:52:59 smithi067 ceph-mon[30939]: Deploying daemon osd.7 on smithi092 2024-09-10T22:53:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:01 smithi092 ceph-mon[34223]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:01.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:01 smithi145 ceph-mon[34397]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:01.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:01 smithi067 ceph-mon[30939]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:03.351 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:03 smithi067 ceph-mon[30939]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:03.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:03 smithi145 ceph-mon[34397]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:03.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:03 smithi092 ceph-mon[34223]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:05.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:05 smithi145 ceph-mon[34397]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:05.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:05 smithi092 ceph-mon[34223]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:05 smithi067 ceph-mon[30939]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:06.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:06.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:06.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:06.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:06.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:06.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:06.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:07.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:07 smithi092 ceph-mon[34223]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:07 smithi145 ceph-mon[34397]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:07 smithi067 ceph-mon[30939]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:09.416 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-09-10T22:53:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.414+0000 7ff5de000640 1 -- 172.21.15.92:0/2448720 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5e0063590 con 0x7ff5c80b53b0 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 msgr2=0x7ff5c80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 0x7ff5c80b7870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400f650 tx=0x7ff5d40075b0 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 msgr2=0x7ff5e019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc02a910 tx=0x7ff5cc003c20 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 shutdown_connections 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff5c80b53b0 0x7ff5c80b7870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e010b330 0x7ff5e01a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e010a9b0 0x7ff5e019b880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 --2- 172.21.15.92:0/2448720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e0104fa0 0x7ff5e019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 >> 172.21.15.92:0/2448720 conn(0x7ff5e0100a50 msgr2=0x7ff5e0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 shutdown_connections 2024-09-10T22:53:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:53:09.417+0000 7ff5e8200640 1 -- 172.21.15.92:0/2448720 wait complete. 2024-09-10T22:53:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:09 smithi092 ceph-mon[34223]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:09.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:09.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:09 smithi145 ceph-mon[34397]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:09 smithi067 ceph-mon[30939]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:10.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:10.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:10.281 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T22:53:10.286 INFO:tasks.cephadm:Deploying osd.8 on smithi145 with /dev/vg_nvme/lv_2... 2024-09-10T22:53:10.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:53:10.660 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:53:11.052 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:10 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.052 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:10 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.206 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:10 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:10 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.302 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:10 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.302 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:10 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:11.771 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 22:53:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T22:53:11.564+0000 7f3c0ee73640 -1 osd.7 0 log_to_monitors true 2024-09-10T22:53:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:11 smithi145 ceph-mon[34397]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:11 smithi145 ceph-mon[34397]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:11 smithi145 ceph-mon[34397]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:12.212 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:11 smithi092 ceph-mon[34223]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:12.212 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:11 smithi092 ceph-mon[34223]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:12.213 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:11 smithi092 ceph-mon[34223]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:11 smithi067 ceph-mon[30939]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:11 smithi067 ceph-mon[30939]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:11 smithi067 ceph-mon[30939]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:53:13.021 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T22:53:12.914+0000 7f3c09a00640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T22:53:13.022 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T22:53:12.928+0000 7f3c04a00640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:53:13.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-mon[34223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:53:13.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-mon[34223]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:13.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-mon[34223]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T22:53:13.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:13.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:12 smithi092 ceph-mon[34223]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:13.162 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:12 smithi145 ceph-mon[34397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:53:13.162 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:12 smithi145 ceph-mon[34397]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:13.162 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:12 smithi145 ceph-mon[34397]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T22:53:13.162 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:13.162 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:12 smithi145 ceph-mon[34397]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:13.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:12 smithi067 ceph-mon[30939]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:53:13.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:12 smithi067 ceph-mon[30939]: from='osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:13.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:12 smithi067 ceph-mon[30939]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T22:53:13.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:13.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:12 smithi067 ceph-mon[30939]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:53:14.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:14.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:53:14.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T22:53:14.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.058 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.059 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] boot 2024-09-10T22:53:14.059 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T22:53:14.059 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:13 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.065 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:53:14.100 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-09-10T22:53:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] boot 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T22:53:14.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:13 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: osd.7 [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] boot 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T22:53:14.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:13 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:53:14.475 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:53:15.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.076+0000 7f7616a00640 1 -- 172.21.15.145:0/866658159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610108fa0 msgr2=0x7f761010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.076+0000 7f7616a00640 1 --2- 172.21.15.145:0/866658159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610108fa0 0x7f761010f7a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ca0 tx=0x7f760802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 -- 172.21.15.145:0/866658159 shutdown_connections 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 --2- 172.21.15.145:0/866658159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610108fa0 0x7f761010f7a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 --2- 172.21.15.145:0/866658159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 0x7f7610108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 --2- 172.21.15.145:0/866658159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610102c10 0x7f7610102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 -- 172.21.15.145:0/866658159 >> 172.21.15.145:0/866658159 conn(0x7f76100fe660 msgr2=0x7f7610100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 -- 172.21.15.145:0/866658159 shutdown_connections 2024-09-10T22:53:15.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.077+0000 7f7616a00640 1 -- 172.21.15.145:0/866658159 wait complete. 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.078+0000 7f7616a00640 1 Processor -- start 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.078+0000 7f7616a00640 1 -- start start 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 0x7f761019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610108fa0 0x7f76101a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7610112b60 con 0x7f7610108fa0 2024-09-10T22:53:15.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76101129e0 con 0x7f7610108620 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7610112ce0 con 0x7f7610102c10 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54894/0 (socket says 172.21.15.145:54894) 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 -- 172.21.15.145:0/1377138635 learned_addr learned my addr 172.21.15.145:0/1377138635 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7614c00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 0x7f761019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7616000640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610108fa0 0x7f76101a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 -- 172.21.15.145:0/1377138635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 msgr2=0x7f761019b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 0x7f761019b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 -- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610108fa0 msgr2=0x7f76101a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:15.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610108fa0 0x7f76101a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:15.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.079+0000 7f7615600640 1 -- 172.21.15.145:0/1377138635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76101a11a0 con 0x7f7610102c10 2024-09-10T22:53:15.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f7615600640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9f0 tx=0x7f760400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:15.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400cd80 con 0x7f7610102c10 2024-09-10T22:53:15.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76040108c0 con 0x7f7610102c10 2024-09-10T22:53:15.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040187c0 con 0x7f7610102c10 2024-09-10T22:53:15.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76101a1490 con 0x7f7610102c10 2024-09-10T22:53:15.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.080+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101a7b50 con 0x7f7610102c10 2024-09-10T22:53:15.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.082+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7610102ff0 con 0x7f7610102c10 2024-09-10T22:53:15.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.084+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f7604010430 con 0x7f7610102c10 2024-09-10T22:53:15.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.086+0000 7f760e000640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 0x7f75f80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:15.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.087+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5612+0+0 (secure 0 0 0) 0x7f76040e5a00 con 0x7f7610102c10 2024-09-10T22:53:15.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.087+0000 7f7614c00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 0x7f75f80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:15.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.087+0000 7f7614c00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 0x7f75f80b7920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7610107fa0 tx=0x7f75fc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:15.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.088+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76040acc60 con 0x7f7610102c10 2024-09-10T22:53:15.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:15.301+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7610063590 con 0x7f75f80b5460 2024-09-10T22:53:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi092 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:15.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:15.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T22:53:15.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi092 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T22:53:15.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:53:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:53:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi092 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T22:53:15.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:16.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T22:53:16.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:16.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:16.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:16.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:16.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:16.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:16.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:17 smithi145 ceph-mon[34397]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:17 smithi092 ceph-mon[34223]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:17 smithi067 ceph-mon[30939]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/4139377622' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:18 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:18 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"}]': finished 2024-09-10T22:53:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:18 smithi145 ceph-mon[34397]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:53:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/4139377622' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:18 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:18 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"}]': finished 2024-09-10T22:53:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:18 smithi092 ceph-mon[34223]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:53:19.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:19.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/4139377622' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:18 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"} : dispatch 2024-09-10T22:53:19.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:18 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8630c3d-7812-4bca-a22d-3539f4fa03d9"}]': finished 2024-09-10T22:53:19.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:18 smithi067 ceph-mon[30939]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:53:19.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:19 smithi145 ceph-mon[34397]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/2311476577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:19 smithi092 ceph-mon[34223]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/2311476577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:19 smithi067 ceph-mon[30939]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/2311476577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:21.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:20 smithi145 ceph-mon[34397]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T22:53:21.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:20 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T22:53:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:20 smithi092 ceph-mon[34223]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T22:53:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:20 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T22:53:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:20 smithi067 ceph-mon[30939]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T22:53:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:20 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T22:53:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:21 smithi145 ceph-mon[34397]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:53:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:21 smithi092 ceph-mon[34223]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:53:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:21 smithi067 ceph-mon[30939]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:53:24.126 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:23 smithi145 ceph-mon[34397]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:53:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:23 smithi092 ceph-mon[34223]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:53:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:23 smithi067 ceph-mon[30939]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:53:26.116 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:25 smithi145 ceph-mon[34397]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-10T22:53:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:25 smithi092 ceph-mon[34223]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-10T22:53:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:25 smithi067 ceph-mon[30939]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-10T22:53:28.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:27 smithi145 ceph-mon[34397]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:28.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:53:28.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:27 smithi092 ceph-mon[34223]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:53:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:28.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:27 smithi067 ceph-mon[30939]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:28.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:53:28.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:29.164 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:28 smithi145 ceph-mon[34397]: Deploying daemon osd.8 on smithi145 2024-09-10T22:53:29.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:28 smithi092 ceph-mon[34223]: Deploying daemon osd.8 on smithi145 2024-09-10T22:53:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:28 smithi067 ceph-mon[30939]: Deploying daemon osd.8 on smithi145 2024-09-10T22:53:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:29 smithi092 ceph-mon[34223]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:30.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:29 smithi067 ceph-mon[30939]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:29 smithi145 ceph-mon[34397]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:53:32.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:31 smithi145 ceph-mon[34397]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T22:53:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:31 smithi092 ceph-mon[34223]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T22:53:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:31 smithi067 ceph-mon[30939]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T22:53:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:33 smithi145 ceph-mon[34397]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:33 smithi092 ceph-mon[34223]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:33 smithi067 ceph-mon[30939]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:36.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:35 smithi145 ceph-mon[34397]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:36.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:36.211 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:35 smithi092 ceph-mon[34223]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:35 smithi067 ceph-mon[30939]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:38.049 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:37 smithi145 ceph-mon[34397]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:37 smithi092 ceph-mon[34223]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:37 smithi067 ceph-mon[30939]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:40.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.175+0000 7f760e000640 1 -- 172.21.15.145:0/1377138635 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7610063590 con 0x7f75f80b5460 2024-09-10T22:53:40.177 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 8 on host 'smithi145' 2024-09-10T22:53:40.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 msgr2=0x7f75f80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 0x7f75f80b7920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7610107fa0 tx=0x7f75fc00b410 comp rx=0 tx=0).stop 2024-09-10T22:53:40.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 msgr2=0x7f761019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9f0 tx=0x7f760400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 shutdown_connections 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75f80b5460 0x7f75f80b7920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7610108fa0 0x7f76101a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7610108620 0x7f761019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 --2- 172.21.15.145:0/1377138635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7610102c10 0x7f761019b0f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 >> 172.21.15.145:0/1377138635 conn(0x7f76100fe660 msgr2=0x7f7610104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 shutdown_connections 2024-09-10T22:53:40.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:53:40.178+0000 7f7616a00640 1 -- 172.21.15.145:0/1377138635 wait complete. 2024-09-10T22:53:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:40.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:40.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.062 DEBUG:teuthology.orchestra.run.smithi145:osd.8> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T22:53:41.065 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2024-09-10T22:53:41.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:53:41.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:41.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.240 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:41.880 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 22:53:41 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T22:53:41.755+0000 7fe01c041640 -1 osd.8 0 log_to_monitors true 2024-09-10T22:53:42.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:41 smithi092 ceph-mon[34223]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:42.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:41 smithi092 ceph-mon[34223]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:42.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:41 smithi092 ceph-mon[34223]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:41 smithi067 ceph-mon[30939]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:41 smithi067 ceph-mon[30939]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:42.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:41 smithi067 ceph-mon[30939]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:42.368 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:41 smithi145 ceph-mon[34397]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:42.368 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:41 smithi145 ceph-mon[34397]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:42.368 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:41 smithi145 ceph-mon[34397]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:53:43.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:42 smithi067 ceph-mon[30939]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:53:43.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:42 smithi067 ceph-mon[30939]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T22:53:43.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:43.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:42 smithi067 ceph-mon[30939]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:42 smithi067 ceph-mon[30939]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:42 smithi092 ceph-mon[34223]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:53:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:42 smithi092 ceph-mon[34223]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T22:53:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:42 smithi092 ceph-mon[34223]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:42 smithi092 ceph-mon[34223]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:42 smithi145 ceph-mon[34397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:53:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:42 smithi145 ceph-mon[34397]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T22:53:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:42 smithi145 ceph-mon[34397]: from='osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:42 smithi145 ceph-mon[34397]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:53:43.839 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T22:53:43.437+0000 7fe016800640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T22:53:43.839 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T22:53:43.460+0000 7fe00c800640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:43 smithi067 ceph-mon[30939]: from='osd.8 ' entity='osd.8' 2024-09-10T22:53:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:43 smithi145 ceph-mon[34397]: from='osd.8 ' entity='osd.8' 2024-09-10T22:53:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:53:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T22:53:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T22:53:44.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:44.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:43 smithi092 ceph-mon[34223]: from='osd.8 ' entity='osd.8' 2024-09-10T22:53:44.815 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:53:44.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-09-10T22:53:45.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:53:45.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:53:45.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] boot 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi145 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:45.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.231 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:53:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:53:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] boot 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi145 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:45.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:45.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:53:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: osd.8 [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] boot 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi145 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:53:45.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-10T22:53:45.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:45.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:45.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:53:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 -- 172.21.15.67:0/2315783160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 msgr2=0x7f75d01119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 --2- 172.21.15.67:0/2315783160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01119a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ca0 tx=0x7f75c802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 -- 172.21.15.67:0/2315783160 shutdown_connections 2024-09-10T22:53:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 --2- 172.21.15.67:0/2315783160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01119a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 --2- 172.21.15.67:0/2315783160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d010a820 0x7f75d010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 --2- 172.21.15.67:0/2315783160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d0104e10 0x7f75d01051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 -- 172.21.15.67:0/2315783160 >> 172.21.15.67:0/2315783160 conn(0x7f75d0100840 msgr2=0x7f75d0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 -- 172.21.15.67:0/2315783160 shutdown_connections 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.826+0000 7f75d4a00640 1 -- 172.21.15.67:0/2315783160 wait complete. 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.827+0000 7f75d4a00640 1 Processor -- start 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.827+0000 7f75d4a00640 1 -- start start 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.827+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 0x7f75d019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.827+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d010a820 0x7f75d019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.827+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75d4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0114d60 con 0x7f75d010b1a0 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75d4a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0114be0 con 0x7f75d0104e10 2024-09-10T22:53:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75d4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0114ee0 con 0x7f75d010a820 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48272/0 (socket says 172.21.15.67:48272) 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 -- 172.21.15.67:0/1585667251 learned_addr learned my addr 172.21.15.67:0/1585667251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75c7e00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d010a820 0x7f75d019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75ce800640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 0x7f75d019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 -- 172.21.15.67:0/1585667251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d010a820 msgr2=0x7f75d019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d010a820 0x7f75d019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 -- 172.21.15.67:0/1585667251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 msgr2=0x7f75d019b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 0x7f75d019b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75cf200640 1 -- 172.21.15.67:0/1585667251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a1200 con 0x7f75d010b1a0 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.828+0000 7f75ce800640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 0x7f75d019b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75cf200640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75c8031040 tx=0x7f75c8032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004040 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a1490 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c8004670 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8030bb0 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.829+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a7bb0 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.830+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d01051f0 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.832+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f75c80041e0 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.834+0000 7f75c7400640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 0x7f75b40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.834+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6044+0+0 (secure 0 0 0) 0x7f75c81066b0 con 0x7f75d010b1a0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.834+0000 7f75ce800640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 0x7f75b40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.835+0000 7f75ce800640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 0x7f75b40b78a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009930 tx=0x7f75c0008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:45.838+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f75c80cddb0 con 0x7f75d010b1a0 2024-09-10T22:53:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:53:46.047+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f75d0063590 con 0x7f75b40b53e0 2024-09-10T22:53:46.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:46 smithi067 ceph-mon[30939]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T22:53:46.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:46.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:46.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:46 smithi145 ceph-mon[34397]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T22:53:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:46.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:46.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:46.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:46 smithi092 ceph-mon[34223]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T22:53:46.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:53:46.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:53:46.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:47.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:47 smithi092 ceph-mon[34223]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:47.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:47 smithi092 ceph-mon[34223]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:47.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:47 smithi092 ceph-mon[34223]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T22:53:47.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:47 smithi067 ceph-mon[30939]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:47.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:47 smithi067 ceph-mon[30939]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:47.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:47 smithi067 ceph-mon[30939]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T22:53:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:47 smithi145 ceph-mon[34397]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:47.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:47 smithi145 ceph-mon[34397]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:53:47.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:47 smithi145 ceph-mon[34397]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T22:53:49.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:49 smithi067 ceph-mon[30939]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:49 smithi145 ceph-mon[34397]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:49 smithi092 ceph-mon[34223]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:53:50.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:50 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"} : dispatch 2024-09-10T22:53:50.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:50 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"}]': finished 2024-09-10T22:53:50.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:50 smithi067 ceph-mon[30939]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T22:53:50.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:50 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:53:50.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:50 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2907577041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:50.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:50 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"} : dispatch 2024-09-10T22:53:50.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:50 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"}]': finished 2024-09-10T22:53:50.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:50 smithi145 ceph-mon[34397]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T22:53:50.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:50 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:53:50.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:50 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2907577041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:50.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:50 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"} : dispatch 2024-09-10T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:50 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2366538735' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c52ab99b-433a-44a6-9fab-dbd35f1bf6d7"}]': finished 2024-09-10T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:50 smithi092 ceph-mon[34223]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:50 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:53:50.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:50 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2907577041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:53:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:51 smithi092 ceph-mon[34223]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:53:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:51 smithi067 ceph-mon[30939]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:53:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:51 smithi145 ceph-mon[34397]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:53:53.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:53 smithi067 ceph-mon[30939]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:53:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:53 smithi092 ceph-mon[34223]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:53:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:53 smithi145 ceph-mon[34397]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:53:55.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:55 smithi067 ceph-mon[30939]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:53:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:55 smithi092 ceph-mon[34223]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:53:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:55 smithi145 ceph-mon[34397]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:53:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:57 smithi067 ceph-mon[30939]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:53:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:57 smithi092 ceph-mon[34223]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:53:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:57 smithi145 ceph-mon[34397]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:53:59.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:59 smithi067 ceph-mon[30939]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:53:59.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:53:59.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:59.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:53:59 smithi067 ceph-mon[30939]: Deploying daemon osd.9 on smithi067 2024-09-10T22:53:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:59 smithi092 ceph-mon[34223]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:53:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:53:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:53:59 smithi092 ceph-mon[34223]: Deploying daemon osd.9 on smithi067 2024-09-10T22:53:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:59 smithi145 ceph-mon[34397]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:53:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:53:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:53:59 smithi145 ceph-mon[34397]: Deploying daemon osd.9 on smithi067 2024-09-10T22:54:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:01 smithi092 ceph-mon[34223]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:01 smithi067 ceph-mon[30939]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:01 smithi145 ceph-mon[34397]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:03 smithi067 ceph-mon[30939]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:03 smithi092 ceph-mon[34223]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:03 smithi145 ceph-mon[34397]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:05.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:05 smithi067 ceph-mon[30939]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:05 smithi092 ceph-mon[34223]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:05 smithi145 ceph-mon[34397]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:07.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:07 smithi067 ceph-mon[30939]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:07.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:07.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:07.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:07 smithi092 ceph-mon[34223]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:07 smithi145 ceph-mon[34397]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:09 smithi092 ceph-mon[34223]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:09 smithi067 ceph-mon[30939]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:09 smithi145 ceph-mon[34397]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:11.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.365+0000 7f75c7400640 1 -- 172.21.15.67:0/1585667251 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75d0063590 con 0x7f75b40b53e0 2024-09-10T22:54:11.367 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2024-09-10T22:54:11.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 msgr2=0x7f75b40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 0x7f75b40b78a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009930 tx=0x7f75c0008040 comp rx=0 tx=0).stop 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 msgr2=0x7f75d01a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75c8031040 tx=0x7f75c8032bc0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 shutdown_connections 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75b40b53e0 0x7f75b40b78a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.367+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75d010b1a0 0x7f75d01a0a30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.368+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75d010a820 0x7f75d019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.368+0000 7f75d4a00640 1 --2- 172.21.15.67:0/1585667251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75d0104e10 0x7f75d019b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.368+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 >> 172.21.15.67:0/1585667251 conn(0x7f75d0100840 msgr2=0x7f75d0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.368+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 shutdown_connections 2024-09-10T22:54:11.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:54:11.368+0000 7f75d4a00640 1 -- 172.21.15.67:0/1585667251 wait complete. 2024-09-10T22:54:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.779 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:11.779 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:12.208 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T22:54:12.213 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-09-10T22:54:12.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:54:12.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:54:12.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:13.049 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 22:54:12 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T22:54:12.940+0000 7ff2c6841640 -1 osd.9 0 log_to_monitors true 2024-09-10T22:54:13.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:13 smithi067 ceph-mon[30939]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:13.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:13 smithi067 ceph-mon[30939]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:54:13.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:13 smithi092 ceph-mon[34223]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:13.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:13 smithi092 ceph-mon[34223]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:54:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:13 smithi145 ceph-mon[34397]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:13 smithi145 ceph-mon[34397]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:54:14.549 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T22:54:14.464+0000 7ff2c1400640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T22:54:14.549 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T22:54:14.480+0000 7ff2bc400640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:54:14.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:54:14.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T22:54:14.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:54:14.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:14.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:54:14.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T22:54:14.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:14.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:54:14.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T22:54:14.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:54:14.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:14.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:54:14.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T22:54:14.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:14.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]} : dispatch 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: from='osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T22:54:14.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:15 smithi145 ceph-mon[34397]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:15 smithi092 ceph-mon[34223]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:15.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:15 smithi067 ceph-mon[30939]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-10T22:54:15.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-09-10T22:54:16.229 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:54:16.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] boot 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: osdmap e62: 10 total, 10 up, 10 in 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:16.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.646 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] boot 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: osdmap e62: 10 total, 10 up, 10 in 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.647 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.648 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.648 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:16.648 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:16.648 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.839+0000 7f06e2400640 1 -- 172.21.15.92:0/1997161707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc109150 msgr2=0x7f06dc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 --2- 172.21.15.92:0/1997161707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc109150 0x7f06dc10f950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009900 tx=0x7f06d402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 -- 172.21.15.92:0/1997161707 shutdown_connections 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 --2- 172.21.15.92:0/1997161707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc109150 0x7f06dc10f950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 --2- 172.21.15.92:0/1997161707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc1087d0 0x7f06dc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 --2- 172.21.15.92:0/1997161707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc102dc0 0x7f06dc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 -- 172.21.15.92:0/1997161707 >> 172.21.15.92:0/1997161707 conn(0x7f06dc0fe830 msgr2=0x7f06dc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 -- 172.21.15.92:0/1997161707 shutdown_connections 2024-09-10T22:54:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.840+0000 7f06e2400640 1 -- 172.21.15.92:0/1997161707 wait complete. 2024-09-10T22:54:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 Processor -- start 2024-09-10T22:54:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 -- start start 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 0x7f06dc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 0x7f06dc1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc112cd0 con 0x7f06dc109150 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc112b50 con 0x7f06dc1087d0 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06e2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc112e50 con 0x7f06dc102dc0 2024-09-10T22:54:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06d2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06d2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36946/0 (socket says 172.21.15.92:36946) 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06d2a00640 1 -- 172.21.15.92:0/1774898635 learned_addr learned my addr 172.21.15.92:0/1774898635 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.841+0000 7f06db400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 0x7f06dc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06dbe00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 0x7f06dc1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 -- 172.21.15.92:0/1774898635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 msgr2=0x7f06dc19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 -- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 msgr2=0x7f06dc1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 0x7f06dc1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 -- 172.21.15.92:0/1774898635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc1a1440 con 0x7f06dc102dc0 2024-09-10T22:54:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06d2a00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06dbe00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 0x7f06dc1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.842+0000 7f06db400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 0x7f06dc19b390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00eb00 tx=0x7f06cc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.843+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc010430 con 0x7f06dc102dc0 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.843+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc1a1730 con 0x7f06dc102dc0 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.843+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06cc010a60 con 0x7f06dc102dc0 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.843+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0179c0 con 0x7f06dc102dc0 2024-09-10T22:54:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.843+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc1a7df0 con 0x7f06dc102dc0 2024-09-10T22:54:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.844+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc1031a0 con 0x7f06dc102dc0 2024-09-10T22:54:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.846+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f06cc017c40 con 0x7f06dc102dc0 2024-09-10T22:54:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.847+0000 7f06d8c00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 0x7f06c00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.848+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6476+0+0 (secure 0 0 0) 0x7f06cc0e6050 con 0x7f06dc102dc0 2024-09-10T22:54:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.848+0000 7f06d2a00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 0x7f06c00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.849+0000 7f06d2a00640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 0x7f06c00b77f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06dc104ff0 tx=0x7f06c4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:16.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:16.849+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06cc0ad630 con 0x7f06dc102dc0 2024-09-10T22:54:16.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: osd.9 [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] boot 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: osdmap e62: 10 total, 10 up, 10 in 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:16.882 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:17.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:17.046+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f06dc063590 con 0x7f06c00b5330 2024-09-10T22:54:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi067 2024-09-10T22:54:17.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-10T22:54:17.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T22:54:17.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:17.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:17.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi067 2024-09-10T22:54:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-10T22:54:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T22:54:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:17.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:17.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:54:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi067 2024-09-10T22:54:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-10T22:54:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T22:54:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:17.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:17.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:18.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:18 smithi092 ceph-mon[34223]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:18.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:18 smithi067 ceph-mon[30939]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:18.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:18 smithi145 ceph-mon[34397]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:19 smithi092 ceph-mon[34223]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:19 smithi067 ceph-mon[30939]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:19 smithi145 ceph-mon[34397]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:20.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/696178035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:20 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:20 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"}]': finished 2024-09-10T22:54:20.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:20 smithi092 ceph-mon[34223]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T22:54:20.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:20 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/696178035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:20 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:20 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"}]': finished 2024-09-10T22:54:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:20 smithi067 ceph-mon[30939]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T22:54:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:20 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:20.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/696178035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:20 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"} : dispatch 2024-09-10T22:54:20.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:20 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96f60cf-4ed2-4c62-bdfd-90c1632a5632"}]': finished 2024-09-10T22:54:20.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:20 smithi145 ceph-mon[34397]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T22:54:20.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:20 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:21 smithi092 ceph-mon[34223]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:21 smithi092 ceph-mon[34223]: from='client.? 172.21.15.92:0/2735847728' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:21 smithi067 ceph-mon[30939]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:21 smithi067 ceph-mon[30939]: from='client.? 172.21.15.92:0/2735847728' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:21 smithi145 ceph-mon[34397]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:21.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:21 smithi145 ceph-mon[34397]: from='client.? 172.21.15.92:0/2735847728' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:23 smithi092 ceph-mon[34223]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:23 smithi067 ceph-mon[30939]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:23 smithi145 ceph-mon[34397]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:25 smithi092 ceph-mon[34223]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:25 smithi067 ceph-mon[30939]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:25 smithi145 ceph-mon[34397]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:27.598 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:27 smithi092 ceph-mon[34223]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:27 smithi067 ceph-mon[30939]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:27 smithi145 ceph-mon[34397]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:29 smithi092 ceph-mon[34223]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:54:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:29 smithi067 ceph-mon[30939]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:54:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:29 smithi145 ceph-mon[34397]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:54:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:30.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:30 smithi092 ceph-mon[34223]: Deploying daemon osd.10 on smithi092 2024-09-10T22:54:30.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:30 smithi067 ceph-mon[30939]: Deploying daemon osd.10 on smithi092 2024-09-10T22:54:30.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:30 smithi145 ceph-mon[34397]: Deploying daemon osd.10 on smithi092 2024-09-10T22:54:31.627 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:31 smithi092 ceph-mon[34223]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:31 smithi067 ceph-mon[30939]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:31 smithi145 ceph-mon[34397]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:33.670 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:33 smithi092 ceph-mon[34223]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:33.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:33 smithi067 ceph-mon[30939]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:33 smithi145 ceph-mon[34397]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:35.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:35 smithi092 ceph-mon[34223]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:35 smithi067 ceph-mon[30939]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:35 smithi145 ceph-mon[34397]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:37 smithi092 ceph-mon[34223]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:37 smithi067 ceph-mon[30939]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:37 smithi145 ceph-mon[34397]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:39 smithi092 ceph-mon[34223]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:39 smithi067 ceph-mon[30939]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:39 smithi145 ceph-mon[34397]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:40.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.876+0000 7f06d8c00640 1 -- 172.21.15.92:0/1774898635 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f06dc063590 con 0x7f06c00b5330 2024-09-10T22:54:40.880 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-09-10T22:54:40.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 msgr2=0x7f06c00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 0x7f06c00b77f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06dc104ff0 tx=0x7f06c4005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:40.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 msgr2=0x7f06dc19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 0x7f06dc19b390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00eb00 tx=0x7f06cc00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 shutdown_connections 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f06c00b5330 0x7f06c00b77f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06dc109150 0x7f06dc1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06dc1087d0 0x7f06dc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 --2- 172.21.15.92:0/1774898635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06dc102dc0 0x7f06dc19b390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 >> 172.21.15.92:0/1774898635 conn(0x7f06dc0fe830 msgr2=0x7f06dc106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 shutdown_connections 2024-09-10T22:54:40.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-10T22:54:40.879+0000 7f06e2400640 1 -- 172.21.15.92:0/1774898635 wait complete. 2024-09-10T22:54:41.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:41.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:41.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:41.855 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T22:54:41.859 INFO:tasks.cephadm:Deploying osd.11 on smithi145 with /dev/vg_nvme/lv_1... 2024-09-10T22:54:41.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:54:42.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:41 smithi092 ceph-mon[34223]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:42.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:42.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:41 smithi145 ceph-mon[34397]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.231 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:54:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:41 smithi067 ceph-mon[30939]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:42.771 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 22:54:42 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T22:54:42.481+0000 7f47a5c2f640 -1 osd.10 0 log_to_monitors true 2024-09-10T22:54:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:42 smithi145 ceph-mon[34397]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:42 smithi145 ceph-mon[34397]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:43.262 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:42 smithi092 ceph-mon[34223]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:43.263 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:42 smithi092 ceph-mon[34223]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:42 smithi067 ceph-mon[30939]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:42 smithi067 ceph-mon[30939]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:54:43.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:43.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:54:43.905 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T22:54:43.905 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:43.905 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:43.905 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:43 smithi145 ceph-mon[34397]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:43.972 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T22:54:43.863+0000 7f47a0400640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T22:54:43.972 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T22:54:43.876+0000 7f479b400640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:54:43.972 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:43.972 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:54:43.973 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T22:54:43.973 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:43.973 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:43.973 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:43 smithi092 ceph-mon[34223]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:44.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:54:44.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T22:54:44.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:44.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: from='osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:44.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:43 smithi067 ceph-mon[30939]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-10T22:54:44.919 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:44 smithi145 ceph-mon[34397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:54:44.919 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:44 smithi145 ceph-mon[34397]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T22:54:44.919 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:44.919 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:44 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:44 smithi092 ceph-mon[34223]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:54:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:44 smithi092 ceph-mon[34223]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T22:54:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:44 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:44 smithi067 ceph-mon[30939]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-10T22:54:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:44 smithi067 ceph-mon[30939]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T22:54:45.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:44 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.833 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] boot 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T22:54:45.862 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:45.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-09-10T22:54:46.235 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] boot 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T22:54:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: osd.10 [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] boot 2024-09-10T22:54:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T22:54:46.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 -- 172.21.15.145:0/4161225244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 msgr2=0x7f9310108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 --2- 172.21.15.145:0/4161225244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f9310108bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc02a410 comp rx=0 tx=0).stop 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 -- 172.21.15.145:0/4161225244 shutdown_connections 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 --2- 172.21.15.145:0/4161225244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310109130 0x7f931010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 --2- 172.21.15.145:0/4161225244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f9310108bf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 --2- 172.21.15.145:0/4161225244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310102da0 0x7f9310103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 -- 172.21.15.145:0/4161225244 >> 172.21.15.145:0/4161225244 conn(0x7f93100fe810 msgr2=0x7f9310100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 -- 172.21.15.145:0/4161225244 shutdown_connections 2024-09-10T22:54:46.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.845+0000 7f9316c00640 1 -- 172.21.15.145:0/4161225244 wait complete. 2024-09-10T22:54:46.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 Processor -- start 2024-09-10T22:54:46.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 -- start start 2024-09-10T22:54:46.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310102da0 0x7f931019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 0x7f93101a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112c60 con 0x7f9310102da0 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112ae0 con 0x7f9310109130 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.846+0000 7f9316c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112de0 con 0x7f93101087b0 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f930f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f930f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42254/0 (socket says 172.21.15.145:42254) 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f930f400640 1 -- 172.21.15.145:0/4191694370 learned_addr learned my addr 172.21.15.145:0/4191694370 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f930fe00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310102da0 0x7f931019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 0x7f93101a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 -- 172.21.15.145:0/4191694370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 msgr2=0x7f931019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 -- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310102da0 msgr2=0x7f931019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310102da0 0x7f931019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f9314c00640 1 -- 172.21.15.145:0/4191694370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93101a1430 con 0x7f9310109130 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.847+0000 7f930f400640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f9314c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 0x7f93101a0c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9308009fd0 tx=0x7f930800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930800ce20 con 0x7f9310109130 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9308005020 con 0x7f9310109130 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93101a1720 con 0x7f9310109130 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308010690 con 0x7f9310109130 2024-09-10T22:54:46.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.848+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a7de0 con 0x7f9310109130 2024-09-10T22:54:46.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.849+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9310103180 con 0x7f9310109130 2024-09-10T22:54:46.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.851+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f9308046b00 con 0x7f9310109130 2024-09-10T22:54:46.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.853+0000 7f930cc00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 0x7f92f40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.853+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6909+0+0 (secure 0 0 0) 0x7f93080e5d90 con 0x7f9310109130 2024-09-10T22:54:46.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.853+0000 7f930fe00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 0x7f92f40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.854+0000 7f930fe00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 0x7f92f40b7870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9300009bd0 tx=0x7f9300005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:46.854+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f93080ad130 con 0x7f9310109130 2024-09-10T22:54:47.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:54:47.052+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9310063590 con 0x7f92f40b53b0 2024-09-10T22:54:47.055 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T22:54:47.055 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi092 2024-09-10T22:54:47.055 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.055 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:47.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi092 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-10T22:54:47.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:47.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:47.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi092 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:47.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:54:47.996 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:47 smithi145 ceph-mon[34397]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:47.997 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:47 smithi145 ceph-mon[34397]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T22:54:47.997 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:47.997 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:47.997 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:48.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:47 smithi067 ceph-mon[30939]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:48.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:47 smithi067 ceph-mon[30939]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T22:54:48.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:48.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:48.214 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:47 smithi092 ceph-mon[34223]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:47 smithi092 ceph-mon[34223]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T22:54:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:54:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:54:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:49.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:48 smithi145 ceph-mon[34397]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:49.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:48 smithi092 ceph-mon[34223]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:49.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:48 smithi067 ceph-mon[30939]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:54:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:49 smithi145 ceph-mon[34397]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:49 smithi092 ceph-mon[34223]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:49 smithi067 ceph-mon[30939]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:54:51.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:50 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/3016633772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.056 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:50 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:50 smithi145 ceph-mon[34397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"}]': finished 2024-09-10T22:54:51.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:50 smithi145 ceph-mon[34397]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T22:54:51.057 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:50 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:54:51.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:50 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/3016633772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:50 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:50 smithi092 ceph-mon[34223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"}]': finished 2024-09-10T22:54:51.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:50 smithi092 ceph-mon[34223]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T22:54:51.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:50 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:54:51.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:50 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/3016633772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:50 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"} : dispatch 2024-09-10T22:54:51.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:50 smithi067 ceph-mon[30939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c1830f-3c31-4764-92c2-ffcd6c11acce"}]': finished 2024-09-10T22:54:51.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:50 smithi067 ceph-mon[30939]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T22:54:51.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:50 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:54:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:51 smithi092 ceph-mon[34223]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:54:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:51 smithi092 ceph-mon[34223]: from='client.? 172.21.15.145:0/666756885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:51 smithi067 ceph-mon[30939]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:54:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:51 smithi067 ceph-mon[30939]: from='client.? 172.21.15.145:0/666756885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:51 smithi145 ceph-mon[34397]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:54:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:51 smithi145 ceph-mon[34397]: from='client.? 172.21.15.145:0/666756885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:54:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:53 smithi092 ceph-mon[34223]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:54:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:53 smithi067 ceph-mon[30939]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:54:54.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:53 smithi145 ceph-mon[34397]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:54:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:55 smithi145 ceph-mon[34397]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:54:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:55 smithi092 ceph-mon[34223]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:54:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:55 smithi067 ceph-mon[30939]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:54:58.249 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:57 smithi145 ceph-mon[34397]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T22:54:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:57 smithi092 ceph-mon[34223]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T22:54:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:57 smithi067 ceph-mon[30939]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T22:55:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:54:59 smithi092 ceph-mon[34223]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:55:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:54:59 smithi067 ceph-mon[30939]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:55:00.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:54:59 smithi145 ceph-mon[34397]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:55:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:55:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:55:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:55:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:02.113 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:01 smithi145 ceph-mon[34397]: Deploying daemon osd.11 on smithi145 2024-09-10T22:55:02.114 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:01 smithi145 ceph-mon[34397]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:01 smithi092 ceph-mon[34223]: Deploying daemon osd.11 on smithi145 2024-09-10T22:55:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:01 smithi092 ceph-mon[34223]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:01 smithi067 ceph-mon[30939]: Deploying daemon osd.11 on smithi145 2024-09-10T22:55:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:01 smithi067 ceph-mon[30939]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:03.955 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:03 smithi145 ceph-mon[34397]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:03 smithi092 ceph-mon[34223]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:03 smithi067 ceph-mon[30939]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:05 smithi092 ceph-mon[34223]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:05 smithi067 ceph-mon[30939]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:05 smithi145 ceph-mon[34397]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:08.178 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:07 smithi145 ceph-mon[34397]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:08.178 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:08.178 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:08.178 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:07 smithi092 ceph-mon[34223]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:07 smithi067 ceph-mon[30939]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:10.024 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:09 smithi145 ceph-mon[34397]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:09 smithi092 ceph-mon[34223]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:09 smithi067 ceph-mon[30939]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:11.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.959+0000 7f930cc00640 1 -- 172.21.15.145:0/4191694370 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9310063590 con 0x7f92f40b53b0 2024-09-10T22:55:11.961 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 11 on host 'smithi145' 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 msgr2=0x7f92f40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 0x7f92f40b7870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9300009bd0 tx=0x7f9300005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 msgr2=0x7f93101a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 0x7f93101a0c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9308009fd0 tx=0x7f930800ef10 comp rx=0 tx=0).stop 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 shutdown_connections 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f92f40b53b0 0x7f92f40b7870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:11.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9310109130 0x7f93101a0c60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:11.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93101087b0 0x7f931019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:11.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 --2- 172.21.15.145:0/4191694370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9310102da0 0x7f931019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:11.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 >> 172.21.15.145:0/4191694370 conn(0x7f93100fe810 msgr2=0x7f9310105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:11.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 shutdown_connections 2024-09-10T22:55:11.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:11.962+0000 7f9316c00640 1 -- 172.21.15.145:0/4191694370 wait complete. 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:12.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:12.908 DEBUG:teuthology.orchestra.run.smithi145:osd.11> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T22:55:12.913 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T22:55:12.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:55:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:13.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.291 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:13.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:13.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:13.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 -- 172.21.15.67:0/1009513732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc077ff0 msgr2=0x7fa8dc113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:13.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 --2- 172.21.15.67:0/1009513732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc113450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009900 tx=0x7fa8c402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:13.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 -- 172.21.15.67:0/1009513732 shutdown_connections 2024-09-10T22:55:13.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 --2- 172.21.15.67:0/1009513732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 --2- 172.21.15.67:0/1009513732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc113450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 --2- 172.21.15.67:0/1009513732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 -- 172.21.15.67:0/1009513732 >> 172.21.15.67:0/1009513732 conn(0x7fa8dc100a50 msgr2=0x7fa8dc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 -- 172.21.15.67:0/1009513732 shutdown_connections 2024-09-10T22:55:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.888+0000 7fa8e2800640 1 -- 172.21.15.67:0/1009513732 wait complete. 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.889+0000 7fa8e2800640 1 Processor -- start 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 -- start start 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc119220 con 0x7fa8dc0776d0 2024-09-10T22:55:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1190a0 con 0x7fa8dc077ff0 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8e2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1193a0 con 0x7fa8dc113990 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.890+0000 7fa8db400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8db400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49106/0 (socket says 172.21.15.67:49106) 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8db400640 1 -- 172.21.15.67:0/417768232 learned_addr learned my addr 172.21.15.67:0/417768232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8dbe00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 -- 172.21.15.67:0/417768232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 msgr2=0x7fa8dc1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc1a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 -- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 msgr2=0x7fa8dc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 -- 172.21.15.67:0/417768232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc1a58a0 con 0x7fa8dc077ff0 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8dbe00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.891+0000 7fa8d2a00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc19fd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005940 tx=0x7fa8c4005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.892+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c403f070 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.892+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8c4003a80 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.892+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4035070 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.892+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc1a5b30 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.892+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1ac250 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.893+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc077ab0 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.895+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fa8c404b060 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.896+0000 7fa8d8c00640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 0x7fa8c00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.897+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7024+0+0 (secure 0 0 0) 0x7fa8c41055c0 con 0x7fa8dc077ff0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.897+0000 7fa8db400640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 0x7fa8c00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.897+0000 7fa8db400640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 0x7fa8c00b77f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009c90 tx=0x7fa8cc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:14.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:13.900+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8c40cc910 con 0x7fa8dc077ff0 2024-09-10T22:55:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:13 smithi092 ceph-mon[34223]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:13 smithi067 ceph-mon[30939]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.099+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa8dc10e590 con 0x7fa8dc077ff0 2024-09-10T22:55:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.100+0000 7fa8d8c00640 1 -- 172.21.15.67:0/417768232 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fa8c40d50c0 con 0x7fa8dc077ff0 2024-09-10T22:55:14.101 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 msgr2=0x7fa8c00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 0x7fa8c00b77f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009c90 tx=0x7fa8cc005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 msgr2=0x7fa8dc19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc19fd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005940 tx=0x7fa8c4005e10 comp rx=0 tx=0).stop 2024-09-10T22:55:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 shutdown_connections 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8c00b5330 0x7fa8c00b77f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8dc113990 0x7fa8dc1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc077ff0 0x7fa8dc19fd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 --2- 172.21.15.67:0/417768232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8dc0776d0 0x7fa8dc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 >> 172.21.15.67:0/417768232 conn(0x7fa8dc100a50 msgr2=0x7fa8dc114590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 shutdown_connections 2024-09-10T22:55:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:14.102+0000 7fa8e2800640 1 -- 172.21.15.67:0/417768232 wait complete. 2024-09-10T22:55:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:13 smithi145 ceph-mon[34397]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:14.131 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 22:55:13 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T22:55:13.993+0000 7efd5660b640 -1 osd.11 0 log_to_monitors true 2024-09-10T22:55:14.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:14 smithi145 ceph-mon[34397]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:14.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:14 smithi145 ceph-mon[34397]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:14.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:14 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/417768232' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:15.104 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726008884,"num_in_osds":12,"osd_in_since":1726008890,"num_remapped_pgs":0} 2024-09-10T22:55:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:14 smithi092 ceph-mon[34223]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:14 smithi092 ceph-mon[34223]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:14 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/417768232' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:14 smithi067 ceph-mon[30939]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:14 smithi067 ceph-mon[30939]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:55:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:14 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/417768232' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:16.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.140 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:15 smithi145 ceph-mon[34397]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.162 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:16.162 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:55:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T22:55:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:15 smithi067 ceph-mon[30939]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: from='osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:15 smithi092 ceph-mon[34223]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]} : dispatch 2024-09-10T22:55:16.406 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T22:55:16.385+0000 7efd51200640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T22:55:16.515 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:16.886 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T22:55:16.404+0000 7efd47200640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:55:17.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:16 smithi067 ceph-mon[30939]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:55:17.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:16 smithi067 ceph-mon[30939]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T22:55:17.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:16 smithi067 ceph-mon[30939]: from='osd.11 ' entity='osd.11' 2024-09-10T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:16 smithi092 ceph-mon[34223]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:16 smithi092 ceph-mon[34223]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:16 smithi092 ceph-mon[34223]: from='osd.11 ' entity='osd.11' 2024-09-10T22:55:17.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 -- 172.21.15.67:0/1118567629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810b350 msgr2=0x7f0a88111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 --2- 172.21.15.67:0/1118567629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810b350 0x7f0a88111b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009ca0 tx=0x7f0a8002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 -- 172.21.15.67:0/1118567629 shutdown_connections 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 --2- 172.21.15.67:0/1118567629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810b350 0x7f0a88111b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 --2- 172.21.15.67:0/1118567629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810a9d0 0x7f0a8810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 --2- 172.21.15.67:0/1118567629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a881053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 -- 172.21.15.67:0/1118567629 >> 172.21.15.67:0/1118567629 conn(0x7f0a881009f0 msgr2=0x7f0a88102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 -- 172.21.15.67:0/1118567629 shutdown_connections 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.281+0000 7f0a90600640 1 -- 172.21.15.67:0/1118567629 wait complete. 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.282+0000 7f0a90600640 1 Processor -- start 2024-09-10T22:55:17.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.282+0000 7f0a90600640 1 -- start start 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 0x7f0a8819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810b350 0x7f0a881a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88114e80 con 0x7f0a8810a9d0 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88114d00 con 0x7f0a8810b350 2024-09-10T22:55:17.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a90600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88115000 con 0x7f0a88104fc0 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8dc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 0x7f0a8819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8dc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:53746/0 (socket says 172.21.15.67:53746) 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8dc00640 1 -- 172.21.15.67:0/373334582 learned_addr learned my addr 172.21.15.67:0/373334582 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 0x7f0a8819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49122/0 (socket says 172.21.15.67:49122) 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.283+0000 7f0a8e600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810b350 0x7f0a881a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 -- 172.21.15.67:0/373334582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810b350 msgr2=0x7f0a881a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810b350 0x7f0a881a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 -- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 msgr2=0x7f0a8819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 0x7f0a8819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 -- 172.21.15.67:0/373334582 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a881a13c0 con 0x7f0a88104fc0 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a8dc00640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00ccc0 con 0x7f0a88104fc0 2024-09-10T22:55:17.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a7c00ce60 con 0x7f0a88104fc0 2024-09-10T22:55:17.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.284+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a881a16b0 con 0x7f0a88104fc0 2024-09-10T22:55:17.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.285+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c010430 con 0x7f0a88104fc0 2024-09-10T22:55:17.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.285+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a881a7d70 con 0x7f0a88104fc0 2024-09-10T22:55:17.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.286+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a881053a0 con 0x7f0a88104fc0 2024-09-10T22:55:17.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.290+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f0a7c004f70 con 0x7f0a88104fc0 2024-09-10T22:55:17.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.291+0000 7f0a86000640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 0x7f0a700b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.292+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7062+0+0 (secure 0 0 0) 0x7f0a7c0e5c00 con 0x7f0a88104fc0 2024-09-10T22:55:17.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.292+0000 7f0a8d200640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 0x7f0a700b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.292+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0a7c0acf00 con 0x7f0a88104fc0 2024-09-10T22:55:17.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.292+0000 7f0a8d200640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 0x7f0a700b77f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a88109fc0 tx=0x7f0a7400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-mon[34397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-10T22:55:17.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-mon[34397]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T22:55:17.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:17.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:16 smithi145 ceph-mon[34397]: from='osd.11 ' entity='osd.11' 2024-09-10T22:55:17.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.491+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0a8802e5c0 con 0x7f0a88104fc0 2024-09-10T22:55:17.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.492+0000 7f0a86000640 1 -- 172.21.15.67:0/373334582 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f0a7c0ac800 con 0x7f0a88104fc0 2024-09-10T22:55:17.494 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.494+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 msgr2=0x7f0a700b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.494+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 0x7f0a700b77f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a88109fc0 tx=0x7f0a7400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 msgr2=0x7f0a8819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00edf0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 shutdown_connections 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0a700b5330 0x7f0a700b77f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8810b350 0x7f0a881a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8810a9d0 0x7f0a8819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 --2- 172.21.15.67:0/373334582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a88104fc0 0x7f0a8819b310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 >> 172.21.15.67:0/373334582 conn(0x7f0a881009f0 msgr2=0x7f0a88102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 shutdown_connections 2024-09-10T22:55:17.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:17.495+0000 7f0a90600640 1 -- 172.21.15.67:0/373334582 wait complete. 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: purged_snaps scrub starts 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: purged_snaps scrub ok 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] boot 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/373334582' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: purged_snaps scrub starts 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: purged_snaps scrub ok 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] boot 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/373334582' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:18.337 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1726008917,"num_in_osds":12,"osd_in_since":1726008890,"num_remapped_pgs":0} 2024-09-10T22:55:18.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:55:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: purged_snaps scrub starts 2024-09-10T22:55:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: purged_snaps scrub ok 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: osd.11 [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] boot 2024-09-10T22:55:18.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:55:18.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:55:18.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/373334582' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:55:18.713 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:19.160 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:18 smithi067 ceph-mon[30939]: Detected new or changed devices on smithi145 2024-09-10T22:55:19.160 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:18 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-10T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:18 smithi092 ceph-mon[34223]: Detected new or changed devices on smithi145 2024-09-10T22:55:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:18 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-10T22:55:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 -- 172.21.15.67:0/3437095078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f4104f90 msgr2=0x7f54f4105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 --2- 172.21.15.67:0/3437095078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f4104f90 0x7f54f4105370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009900 tx=0x7f54e402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 -- 172.21.15.67:0/3437095078 shutdown_connections 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 --2- 172.21.15.67:0/3437095078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410b320 0x7f54f4111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 --2- 172.21.15.67:0/3437095078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f410a9a0 0x7f54f410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 --2- 172.21.15.67:0/3437095078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f4104f90 0x7f54f4105370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 -- 172.21.15.67:0/3437095078 >> 172.21.15.67:0/3437095078 conn(0x7f54f4100a00 msgr2=0x7f54f4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 -- 172.21.15.67:0/3437095078 shutdown_connections 2024-09-10T22:55:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.271+0000 7f54fa400640 1 -- 172.21.15.67:0/3437095078 wait complete. 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.272+0000 7f54fa400640 1 Processor -- start 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.272+0000 7f54fa400640 1 -- start start 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f4104f90 0x7f54f419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410a9a0 0x7f54f419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4114e50 con 0x7f54f410b320 2024-09-10T22:55:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4114cd0 con 0x7f54f4104f90 2024-09-10T22:55:19.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54fa400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4114fd0 con 0x7f54f410a9a0 2024-09-10T22:55:19.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:19.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42562/0 (socket says 172.21.15.67:42562) 2024-09-10T22:55:19.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54f3e00640 1 -- 172.21.15.67:0/3827482195 learned_addr learned my addr 172.21.15.67:0/3827482195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:19.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54f3400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f4104f90 0x7f54f419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:19.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.273+0000 7f54f2a00640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410a9a0 0x7f54f419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:19.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 -- 172.21.15.67:0/3827482195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410a9a0 msgr2=0x7f54f419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:19.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410a9a0 0x7f54f419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 -- 172.21.15.67:0/3827482195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f4104f90 msgr2=0x7f54f419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f4104f90 0x7f54f419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 -- 172.21.15.67:0/3827482195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f41a1490 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54f3e00640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00eaa0 tx=0x7f54ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cdb0 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54ec005020 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0116d0 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.274+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f41a1780 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.275+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f41a7e40 con 0x7f54f410b320 2024-09-10T22:55:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.276+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f4105370 con 0x7f54f410b320 2024-09-10T22:55:19.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.278+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f54ec00f040 con 0x7f54f410b320 2024-09-10T22:55:19.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.279+0000 7f54ebe00640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 0x7f54d80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:19.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.280+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f54ec0e5980 con 0x7f54f410b320 2024-09-10T22:55:19.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.280+0000 7f54f3400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 0x7f54d80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:19.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.280+0000 7f54f3400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 0x7f54d80b77f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e4006010 tx=0x7f54e4005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:19.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.281+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f54ec016030 con 0x7f54f410b320 2024-09-10T22:55:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:18 smithi145 ceph-mon[34397]: Detected new or changed devices on smithi145 2024-09-10T22:55:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:18 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-10T22:55:19.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.476+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54f402e5c0 con 0x7f54f410b320 2024-09-10T22:55:19.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.478+0000 7f54ebe00640 1 -- 172.21.15.67:0/3827482195 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18744 (secure 0 0 0) 0x7f54ec0ac5c0 con 0x7f54f410b320 2024-09-10T22:55:19.480 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:19.480 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:55:18.889764+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.480+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 msgr2=0x7f54d80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.480+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 0x7f54d80b77f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e4006010 tx=0x7f54e4005e80 comp rx=0 tx=0).stop 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.480+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 msgr2=0x7f54f41a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.480+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00eaa0 tx=0x7f54ec00ef60 comp rx=0 tx=0).stop 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 shutdown_connections 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f54d80b5330 0x7f54d80b77f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f410b320 0x7f54f41a0cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54f410a9a0 0x7f54f419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 --2- 172.21.15.67:0/3827482195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54f4104f90 0x7f54f419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 >> 172.21.15.67:0/3827482195 conn(0x7f54f4100a00 msgr2=0x7f54f41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 shutdown_connections 2024-09-10T22:55:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:19.481+0000 7f54fa400640 1 -- 172.21.15.67:0/3827482195 wait complete. 2024-09-10T22:55:20.231 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:19 smithi067 ceph-mon[30939]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:20.231 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:19 smithi067 ceph-mon[30939]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:55:20.231 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3827482195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:55:20.270 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:50:44.441563+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:55:20.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T22:55:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:19 smithi092 ceph-mon[34223]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:19 smithi092 ceph-mon[34223]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:55:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3827482195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:55:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:19 smithi145 ceph-mon[34397]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:19 smithi145 ceph-mon[34397]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:55:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3827482195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:55:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:21.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.340+0000 7fb126000640 1 -- 172.21.15.67:0/1168313227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 msgr2=0x7fb120111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:21.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.340+0000 7fb126000640 1 --2- 172.21.15.67:0/1168313227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb120111780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb118009900 tx=0x7fb11802a410 comp rx=0 tx=0).stop 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 -- 172.21.15.67:0/1168313227 shutdown_connections 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 --2- 172.21.15.67:0/1168313227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb120111780 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 --2- 172.21.15.67:0/1168313227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb120075e00 0x7fb12010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 --2- 172.21.15.67:0/1168313227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1200754e0 0x7fb1200758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 -- 172.21.15.67:0/1168313227 >> 172.21.15.67:0/1168313227 conn(0x7fb1200fe810 msgr2=0x7fb120100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 -- 172.21.15.67:0/1168313227 shutdown_connections 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 -- 172.21.15.67:0/1168313227 wait complete. 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.341+0000 7fb126000640 1 Processor -- start 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 -- start start 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1200754e0 0x7fb12019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb120075e00 0x7fb12019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:21.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb120114e10 con 0x7fb12010f390 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb120114c90 con 0x7fb120075e00 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb126000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb120114f90 con 0x7fb1200754e0 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb11fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb11fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42580/0 (socket says 172.21.15.67:42580) 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.342+0000 7fb11fe00640 1 -- 172.21.15.67:0/2196482232 learned_addr learned my addr 172.21.15.67:0/2196482232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11f400640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1200754e0 0x7fb12019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb116a00640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb120075e00 0x7fb12019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 -- 172.21.15.67:0/2196482232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1200754e0 msgr2=0x7fb12019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1200754e0 0x7fb12019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 -- 172.21.15.67:0/2196482232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb120075e00 msgr2=0x7fb12019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb120075e00 0x7fb12019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 -- 172.21.15.67:0/2196482232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1201a1410 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11fe00640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1180057c0 tx=0x7fb118004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11803f030 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.343+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb118003a80 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.344+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118035070 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.344+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201a16a0 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.344+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201a7dc0 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.345+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1200758c0 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.347+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fb11803bba0 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.348+0000 7fb11cc00640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 0x7fb1040b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.348+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fb1181065f0 con 0x7fb12010f390 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.349+0000 7fb11f400640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 0x7fb1040b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.349+0000 7fb11f400640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 0x7fb1040b7870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb110005fd0 tx=0x7fb110009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:21.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.351+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1180cd190 con 0x7fb12010f390 2024-09-10T22:55:21.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.544+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1201a8080 con 0x7fb12010f390 2024-09-10T22:55:21.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.545+0000 7fb11cc00640 1 -- 172.21.15.67:0/2196482232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fb1180d1bc0 con 0x7fb12010f390 2024-09-10T22:55:21.547 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 msgr2=0x7fb1040b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 0x7fb1040b7870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb110005fd0 tx=0x7fb110009760 comp rx=0 tx=0).stop 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 msgr2=0x7fb1201a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1180057c0 tx=0x7fb118004270 comp rx=0 tx=0).stop 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 shutdown_connections 2024-09-10T22:55:21.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1040b53b0 0x7fb1040b7870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb12010f390 0x7fb1201a0c40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb120075e00 0x7fb12019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 --2- 172.21.15.67:0/2196482232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1200754e0 0x7fb12019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 >> 172.21.15.67:0/2196482232 conn(0x7fb1200fe810 msgr2=0x7fb1200ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 shutdown_connections 2024-09-10T22:55:21.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:21.547+0000 7fb126000640 1 -- 172.21.15.67:0/2196482232 wait complete. 2024-09-10T22:55:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:21 smithi092 ceph-mon[34223]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:21 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2196482232' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:55:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:21 smithi067 ceph-mon[30939]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:21 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2196482232' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:55:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:21 smithi145 ceph-mon[34397]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:21 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2196482232' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:55:22.437 INFO:tasks.cephadm:Adding mds.a on smithi067 2024-09-10T22:55:22.437 INFO:tasks.cephadm:Adding mds.d on smithi067 2024-09-10T22:55:22.437 INFO:tasks.cephadm:Adding mds.g on smithi067 2024-09-10T22:55:22.437 INFO:tasks.cephadm:Adding mds.j on smithi067 2024-09-10T22:55:22.437 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.c on smithi145 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.f on smithi145 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.i on smithi145 2024-09-10T22:55:22.438 INFO:tasks.cephadm:Adding mds.l on smithi145 2024-09-10T22:55:22.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l' 2024-09-10T22:55:22.814 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:55:23.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.395+0000 7f852c600640 1 -- 172.21.15.145:0/3846969656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85241087d0 msgr2=0x7f8524108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.395+0000 7f852c600640 1 --2- 172.21.15.145:0/3846969656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85241087d0 0x7f8524108c10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8510009900 tx=0x7f851002a410 comp rx=0 tx=0).stop 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 -- 172.21.15.145:0/3846969656 shutdown_connections 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 --2- 172.21.15.145:0/3846969656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524109150 0x7f852410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 --2- 172.21.15.145:0/3846969656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85241087d0 0x7f8524108c10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 --2- 172.21.15.145:0/3846969656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524102dc0 0x7f85241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 -- 172.21.15.145:0/3846969656 >> 172.21.15.145:0/3846969656 conn(0x7f85240fe830 msgr2=0x7f8524100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 -- 172.21.15.145:0/3846969656 shutdown_connections 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.396+0000 7f852c600640 1 -- 172.21.15.145:0/3846969656 wait complete. 2024-09-10T22:55:23.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 Processor -- start 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 -- start start 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524102dc0 0x7f852419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85241087d0 0x7f852419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112d10 con 0x7f8524102dc0 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112b90 con 0x7f85241087d0 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112e90 con 0x7f8524109150 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38838/0 (socket says 172.21.15.145:38838) 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f852a600640 1 -- 172.21.15.145:0/657542960 learned_addr learned my addr 172.21.15.145:0/657542960 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T22:55:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.397+0000 7f8529c00640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524102dc0 0x7f852419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f8529200640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85241087d0 0x7f852419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 -- 172.21.15.145:0/657542960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85241087d0 msgr2=0x7f852419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85241087d0 0x7f852419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 -- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524102dc0 msgr2=0x7f852419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524102dc0 0x7f852419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 -- 172.21.15.145:0/657542960 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241a14b0 con 0x7f8524109150 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852a600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d750 tx=0x7f851c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f8524109150 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f851c0054b0 con 0x7f8524109150 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241a17a0 con 0x7f8524109150 2024-09-10T22:55:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.398+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c004b90 con 0x7f8524109150 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.399+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a7e60 con 0x7f8524109150 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.400+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85241031a0 con 0x7f8524109150 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.402+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f851c004d30 con 0x7f8524109150 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.404+0000 7f8522000640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 0x7f850c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.405+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f851c0e55e0 con 0x7f8524109150 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.405+0000 7f8529c00640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 0x7f850c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.405+0000 7f8529c00640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 0x7f850c0b7870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8518005fd0 tx=0x7f8518009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.406+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f851c0ac200 con 0x7f8524109150 2024-09-10T22:55:23.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.601+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}) -- 0x7f8524063590 con 0x7f850c0b53b0 2024-09-10T22:55:23.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.610+0000 7f8522000640 1 -- 172.21.15.145:0/657542960 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8524063590 con 0x7f850c0b53b0 2024-09-10T22:55:23.612 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mds.all update... 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.612+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 msgr2=0x7f850c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.612+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 0x7f850c0b7870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8518005fd0 tx=0x7f8518009680 comp rx=0 tx=0).stop 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.612+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 msgr2=0x7f85241a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.612+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d750 tx=0x7f851c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 shutdown_connections 2024-09-10T22:55:23.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f850c0b53b0 0x7f850c0b7870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8524109150 0x7f85241a0ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85241087d0 0x7f852419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 --2- 172.21.15.145:0/657542960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8524102dc0 0x7f852419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 >> 172.21.15.145:0/657542960 conn(0x7f85240fe830 msgr2=0x7f8524104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 shutdown_connections 2024-09-10T22:55:23.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T22:55:23.613+0000 7f852c600640 1 -- 172.21.15.145:0/657542960 wait complete. 2024-09-10T22:55:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:23 smithi092 ceph-mon[34223]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:23 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:23 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:23 smithi067 ceph-mon[30939]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:23 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:23 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:23 smithi145 ceph-mon[34397]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:23 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:23 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:24.639 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T22:55:24.644 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T22:55:24.688 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T22:55:24.733 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T22:55:24.778 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T22:55:24.782 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T22:55:24.827 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T22:55:24.872 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T22:55:24.917 DEBUG:teuthology.orchestra.run.smithi145:mds.c> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T22:55:24.921 DEBUG:teuthology.orchestra.run.smithi145:mds.f> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T22:55:24.966 DEBUG:teuthology.orchestra.run.smithi145:mds.i> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T22:55:25.011 DEBUG:teuthology.orchestra.run.smithi145:mds.l> sudo journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T22:55:25.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:25.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.056 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T22:55:25.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:25.260 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:25.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:55:25.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-10T22:55:25.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:25.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:25.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:25.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:25.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:25.428 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:26.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:25 smithi067 ceph-mon[30939]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:26.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:25 smithi067 ceph-mon[30939]: Deploying daemon mds.l on smithi145 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 -- 172.21.15.67:0/3593475345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075540 msgr2=0x7fb358075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 --2- 172.21.15.67:0/3593475345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075540 0x7fb358075920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb348009860 tx=0x7fb34802a340 comp rx=0 tx=0).stop 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 -- 172.21.15.67:0/3593475345 shutdown_connections 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 --2- 172.21.15.67:0/3593475345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 --2- 172.21.15.67:0/3593475345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075f40 0x7fb35810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 --2- 172.21.15.67:0/3593475345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075540 0x7fb358075920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 -- 172.21.15.67:0/3593475345 >> 172.21.15.67:0/3593475345 conn(0x7fb3580fe830 msgr2=0x7fb358100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 -- 172.21.15.67:0/3593475345 shutdown_connections 2024-09-10T22:55:26.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.131+0000 7fb35cc00640 1 -- 172.21.15.67:0/3593475345 wait complete. 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 Processor -- start 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 -- start start 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 0x7fb35819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075f40 0x7fb35819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb358114e20 con 0x7fb35810f3e0 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb358114ca0 con 0x7fb358075540 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.132+0000 7fb35cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb358114fa0 con 0x7fb358075f40 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42592/0 (socket says 172.21.15.67:42592) 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 -- 172.21.15.67:0/2433792077 learned_addr learned my addr 172.21.15.67:0/2433792077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb34d400640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075f40 0x7fb35819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb355e00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 0x7fb35819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 -- 172.21.15.67:0/2433792077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075f40 msgr2=0x7fb35819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075f40 0x7fb35819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 -- 172.21.15.67:0/2433792077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 msgr2=0x7fb35819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 0x7fb35819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb356800640 1 -- 172.21.15.67:0/2433792077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581a15c0 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.133+0000 7fb355e00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 0x7fb35819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb356800640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c3f0 tx=0x7fb35000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350003970 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb35000b690 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350004310 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3581a18b0 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.134+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a7f70 con 0x7fb35810f3e0 2024-09-10T22:55:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.135+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb358075920 con 0x7fb35810f3e0 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.137+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fb35000b040 con 0x7fb35810f3e0 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.139+0000 7fb34f400640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 0x7fb33c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.139+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fb3500e5810 con 0x7fb35810f3e0 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.139+0000 7fb355e00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 0x7fb33c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.140+0000 7fb355e00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 0x7fb33c0b77f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb348009830 tx=0x7fb348000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.142+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb350054220 con 0x7fb35810f3e0 2024-09-10T22:55:26.206 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:25 smithi145 ceph-mon[34397]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:26.206 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:25 smithi145 ceph-mon[34397]: Deploying daemon mds.l on smithi145 2024-09-10T22:55:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:25 smithi092 ceph-mon[34223]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:25 smithi092 ceph-mon[34223]: Deploying daemon mds.l on smithi145 2024-09-10T22:55:26.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.378+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35819c640 con 0x7fb35810f3e0 2024-09-10T22:55:26.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.379+0000 7fb34f400640 1 -- 172.21.15.67:0/2433792077 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fb3500acaa0 con 0x7fb35810f3e0 2024-09-10T22:55:26.381 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:26.381 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"btime":"2024-09-10T22:46:22:380568+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T22:55:26.381 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 1 2024-09-10T22:55:26.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 msgr2=0x7fb33c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:26.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 0x7fb33c0b77f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb348009830 tx=0x7fb348000bc0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 msgr2=0x7fb3581a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:26.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c3f0 tx=0x7fb35000c8b0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 shutdown_connections 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb33c0b5330 0x7fb33c0b77f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb35810f3e0 0x7fb3581a0df0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb358075f40 0x7fb35819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.381+0000 7fb35cc00640 1 --2- 172.21.15.67:0/2433792077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb358075540 0x7fb35819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.382+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 >> 172.21.15.67:0/2433792077 conn(0x7fb3580fe830 msgr2=0x7fb3580ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.382+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 shutdown_connections 2024-09-10T22:55:26.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:26.382+0000 7fb35cc00640 1 -- 172.21.15.67:0/2433792077 wait complete. 2024-09-10T22:55:26.973 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:26 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2433792077' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:26 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2433792077' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:27.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:26 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2433792077' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:27.286 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:26 smithi145 systemd[1]: Starting Ceph mds.l for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:27.572 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 podman[65755]: 2024-09-10 22:55:27.189198027 +0000 UTC m=+0.027186343 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:27.572 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 podman[65755]: 2024-09-10 22:55:27.29332525 +0000 UTC m=+0.131313550 container create 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T22:55:27.572 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 podman[65755]: 2024-09-10 22:55:27.571345566 +0000 UTC m=+0.409333870 container init 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T22:55:27.880 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 podman[65755]: 2024-09-10 22:55:27.578056164 +0000 UTC m=+0.416044459 container start 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T22:55:27.880 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mds[65774]: -- 172.21.15.145:0/513848519 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a880701860 con 0x55a8806df400 2024-09-10T22:55:27.880 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l[65770]: starting mds.l at 2024-09-10T22:55:27.880 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 bash[65755]: 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf 2024-09-10T22:55:27.881 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 22:55:27 smithi145 systemd[1]: Started Ceph mds.l for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6832/482762321,v1:172.21.15.145:6833/482762321] up:boot 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: fsmap 1 up:standby 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:28.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:27 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6832/482762321,v1:172.21.15.145:6833/482762321] up:boot 2024-09-10T22:55:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: fsmap 1 up:standby 2024-09-10T22:55:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:55:28.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:28.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:28.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:27 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6832/482762321,v1:172.21.15.145:6833/482762321] up:boot 2024-09-10T22:55:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: fsmap 1 up:standby 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:28.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:27 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:29.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:28 smithi092 ceph-mon[34223]: Deploying daemon mds.k on smithi092 2024-09-10T22:55:29.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:28 smithi067 ceph-mon[30939]: Deploying daemon mds.k on smithi092 2024-09-10T22:55:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:28 smithi145 ceph-mon[34397]: Deploying daemon mds.k on smithi092 2024-09-10T22:55:29.671 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:30.067 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:29 smithi092 ceph-mon[34223]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:30.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:29 smithi067 ceph-mon[30939]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 -- 172.21.15.67:0/3345914048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075f40 msgr2=0x7f9f1410edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 --2- 172.21.15.67:0/3345914048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075f40 0x7f9f1410edf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009900 tx=0x7f9efc02a410 comp rx=0 tx=0).stop 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 -- 172.21.15.67:0/3345914048 shutdown_connections 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 --2- 172.21.15.67:0/3345914048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 0x7f9f141117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 --2- 172.21.15.67:0/3345914048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075f40 0x7f9f1410edf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 --2- 172.21.15.67:0/3345914048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075540 0x7f9f14075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 -- 172.21.15.67:0/3345914048 >> 172.21.15.67:0/3345914048 conn(0x7f9f140fe810 msgr2=0x7f9f14100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 -- 172.21.15.67:0/3345914048 shutdown_connections 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.273+0000 7f9f1a600640 1 -- 172.21.15.67:0/3345914048 wait complete. 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 Processor -- start 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 -- start start 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075f40 0x7f9f1419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 0x7f9f141a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14114e10 con 0x7f9f14075540 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14114c90 con 0x7f9f1410f3c0 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.274+0000 7f9f1a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14114f90 con 0x7f9f14075f40 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59580/0 (socket says 172.21.15.67:59580) 2024-09-10T22:55:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 -- 172.21.15.67:0/4252059719 learned_addr learned my addr 172.21.15.67:0/4252059719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13e00640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 0x7f9f141a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f0aa00640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075f40 0x7f9f1419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 -- 172.21.15.67:0/4252059719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075f40 msgr2=0x7f9f1419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075f40 0x7f9f1419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 -- 172.21.15.67:0/4252059719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 msgr2=0x7f9f141a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 0x7f9f141a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.275+0000 7f9f13400640 1 -- 172.21.15.67:0/4252059719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f141a1470 con 0x7f9f14075540 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f13400640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9f0 tx=0x7f9f0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400cd80 con 0x7f9f14075540 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f040108c0 con 0x7f9f14075540 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f141a1760 con 0x7f9f14075540 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f040187c0 con 0x7f9f14075540 2024-09-10T22:55:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.276+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f141a7e20 con 0x7f9f14075540 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.277+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f14075920 con 0x7f9f14075540 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.279+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f9f04010430 con 0x7f9f14075540 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.281+0000 7f9f10c00640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 0x7f9ef80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.281+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f9f040e6140 con 0x7f9f14075540 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.281+0000 7f9f0aa00640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 0x7f9ef80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.282+0000 7f9f0aa00640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 0x7f9ef80b77f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000c00 tx=0x7f9efc005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.283+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9f040ad3d0 con 0x7f9f14075540 2024-09-10T22:55:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:29 smithi145 ceph-mon[34397]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:30.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.515+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1402e5c0 con 0x7f9f14075540 2024-09-10T22:55:30.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.516+0000 7f9f10c00640 1 -- 172.21.15.67:0/4252059719 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f9f040accd0 con 0x7f9f14075540 2024-09-10T22:55:30.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:30.518 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"btime":"2024-09-10T22:55:27:623420+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:55:30.518 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-09-10T22:55:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 msgr2=0x7f9ef80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 0x7f9ef80b77f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000c00 tx=0x7f9efc005a20 comp rx=0 tx=0).stop 2024-09-10T22:55:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 msgr2=0x7f9f1419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9f0 tx=0x7f9f0400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 shutdown_connections 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9ef80b5330 0x7f9ef80b77f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1410f3c0 0x7f9f141a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f14075f40 0x7f9f1419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 --2- 172.21.15.67:0/4252059719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f14075540 0x7f9f1419b3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 >> 172.21.15.67:0/4252059719 conn(0x7f9f140fe810 msgr2=0x7f9f140ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 shutdown_connections 2024-09-10T22:55:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:30.518+0000 7f9f1a600640 1 -- 172.21.15.67:0/4252059719 wait complete. 2024-09-10T22:55:30.936 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:30 smithi092 podman[64531]: 2024-09-10 22:55:30.698670925 +0000 UTC m=+0.160706850 container create ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:55:31.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:30 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4252059719' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:31.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:30 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4252059719' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:30 smithi092 podman[64531]: 2024-09-10 22:55:30.959965201 +0000 UTC m=+0.422001127 container init ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:30 smithi092 podman[64531]: 2024-09-10 22:55:30.967786431 +0000 UTC m=+0.429822357 container start ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mds[64557]: -- 172.21.15.92:0/3850240746 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f65815c20 con 0x557f657f2800 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: starting mds.k at 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:31 smithi092 bash[64531]: ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a 2024-09-10T22:55:31.272 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 22:55:31 smithi092 systemd[1]: Started Ceph mds.k for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:30 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4252059719' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:32.217 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:32.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:33.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:32 smithi067 ceph-mon[30939]: Deploying daemon mds.j on smithi067 2024-09-10T22:55:33.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:32 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:boot 2024-09-10T22:55:33.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:32 smithi067 ceph-mon[30939]: fsmap 2 up:standby 2024-09-10T22:55:33.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:55:33.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:32 smithi092 ceph-mon[34223]: Deploying daemon mds.j on smithi067 2024-09-10T22:55:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:32 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:boot 2024-09-10T22:55:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:32 smithi092 ceph-mon[34223]: fsmap 2 up:standby 2024-09-10T22:55:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:55:33.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:32 smithi145 ceph-mon[34397]: Deploying daemon mds.j on smithi067 2024-09-10T22:55:33.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:32 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:boot 2024-09-10T22:55:33.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:32 smithi145 ceph-mon[34397]: fsmap 2 up:standby 2024-09-10T22:55:33.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:55:33.864 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:34.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:33 smithi067 ceph-mon[30939]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:33 smithi092 ceph-mon[34223]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:33 smithi145 ceph-mon[34397]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.452+0000 7feff3200640 1 -- 172.21.15.67:0/2458728005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 msgr2=0x7feff4111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.452+0000 7feff3200640 1 --2- 172.21.15.67:0/2458728005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff4111690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fefec009ca0 tx=0x7fefec02a950 comp rx=0 tx=0).stop 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 -- 172.21.15.67:0/2458728005 shutdown_connections 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 --2- 172.21.15.67:0/2458728005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff4111690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 --2- 172.21.15.67:0/2458728005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 0x7feff410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 --2- 172.21.15.67:0/2458728005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 -- 172.21.15.67:0/2458728005 >> 172.21.15.67:0/2458728005 conn(0x7feff40fd560 msgr2=0x7feff40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 -- 172.21.15.67:0/2458728005 shutdown_connections 2024-09-10T22:55:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.453+0000 7feff3200640 1 -- 172.21.15.67:0/2458728005 wait complete. 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.454+0000 7feff3200640 1 Processor -- start 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.454+0000 7feff3200640 1 -- start start 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 0x7feff419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff41a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feff4114c90 con 0x7feff410f2a0 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feff4114b10 con 0x7feff4102010 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feff4114e10 con 0x7feff41016f0 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff41a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:39890/0 (socket says 172.21.15.67:39890) 2024-09-10T22:55:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff1e00640 1 -- 172.21.15.67:0/2354324389 learned_addr learned my addr 172.21.15.67:0/2354324389 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7fefe9400640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 0x7feff419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 -- 172.21.15.67:0/2354324389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 msgr2=0x7feff419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 -- 172.21.15.67:0/2354324389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 msgr2=0x7feff419b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 0x7feff419b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff2800640 1 -- 172.21.15.67:0/2354324389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff41a11b0 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.455+0000 7feff1e00640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7feff2800640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff41a09e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefec02ae70 tx=0x7fefec003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec034040 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefec030850 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec0045e0 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff41a1440 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.456+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff41a7b60 con 0x7feff410f2a0 2024-09-10T22:55:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.457+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff4101ad0 con 0x7feff410f2a0 2024-09-10T22:55:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.459+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fefec04d030 con 0x7feff410f2a0 2024-09-10T22:55:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.461+0000 7fefeb400640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 0x7fefd80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.461+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fefec106540 con 0x7feff410f2a0 2024-09-10T22:55:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.461+0000 7feff1e00640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 0x7fefd80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.462+0000 7feff1e00640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 0x7fefd80b78a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fefe4011500 tx=0x7fefe4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.465+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fefec0098b0 con 0x7feff410f2a0 2024-09-10T22:55:34.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.703+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff41a7d90 con 0x7feff410f2a0 2024-09-10T22:55:34.706 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:34 smithi067 podman[73976]: 2024-09-10 22:55:34.573531666 +0000 UTC m=+0.028038350 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.704+0000 7fefeb400640 1 -- 172.21.15.67:0/2354324389 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2257 (secure 0 0 0) 0x7fefec0cd730 con 0x7feff410f2a0 2024-09-10T22:55:34.706 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:34.706 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"btime":"2024-09-10T22:55:31:934513+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:55:34.707 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-09-10T22:55:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.706+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 msgr2=0x7fefd80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.706+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 0x7fefd80b78a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fefe4011500 tx=0x7fefe4005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.706+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 msgr2=0x7feff41a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.706+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff41a09e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefec02ae70 tx=0x7fefec003c90 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 shutdown_connections 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fefd80b53e0 0x7fefd80b78a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff410f2a0 0x7feff41a09e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff4102010 0x7feff419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 --2- 172.21.15.67:0/2354324389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feff41016f0 0x7feff419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 >> 172.21.15.67:0/2354324389 conn(0x7feff40fd560 msgr2=0x7feff4111540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 shutdown_connections 2024-09-10T22:55:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:34.707+0000 7feff3200640 1 -- 172.21.15.67:0/2354324389 wait complete. 2024-09-10T22:55:35.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:34 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2354324389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:35.049 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:34 smithi067 podman[73976]: 2024-09-10 22:55:34.731947082 +0000 UTC m=+0.186453766 container create 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T22:55:35.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:34 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2354324389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:35.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:34 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2354324389' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:35.455 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 podman[73976]: 2024-09-10 22:55:35.329310382 +0000 UTC m=+0.783817069 container init 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T22:55:35.455 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 podman[73976]: 2024-09-10 22:55:35.335633768 +0000 UTC m=+0.790140460 container start 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:55:35.455 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mds[74008]: -- 172.21.15.67:0/1805340386 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55695c499860 con 0x55695c476800 2024-09-10T22:55:35.455 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j[74003]: starting mds.j at 2024-09-10T22:55:35.798 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 bash[73976]: 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 2024-09-10T22:55:35.798 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 22:55:35 smithi067 systemd[1]: Started Ceph mds.j for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:36.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:36.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:35 smithi067 ceph-mon[30939]: Deploying daemon mds.d on smithi067 2024-09-10T22:55:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:36.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:35 smithi092 ceph-mon[34223]: Deploying daemon mds.d on smithi067 2024-09-10T22:55:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:36.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:35 smithi145 ceph-mon[34397]: Deploying daemon mds.d on smithi067 2024-09-10T22:55:37.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:36 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.67:6834/99494350,v1:172.21.15.67:6835/99494350] up:boot 2024-09-10T22:55:37.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:36 smithi092 ceph-mon[34223]: fsmap 3 up:standby 2024-09-10T22:55:37.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:55:37.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:36 smithi092 ceph-mon[34223]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:37.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:36 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.67:6834/99494350,v1:172.21.15.67:6835/99494350] up:boot 2024-09-10T22:55:37.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:36 smithi067 ceph-mon[30939]: fsmap 3 up:standby 2024-09-10T22:55:37.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:55:37.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:36 smithi067 ceph-mon[30939]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:36 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.67:6834/99494350,v1:172.21.15.67:6835/99494350] up:boot 2024-09-10T22:55:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:36 smithi145 ceph-mon[34397]: fsmap 3 up:standby 2024-09-10T22:55:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:55:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:36 smithi145 ceph-mon[34397]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:37.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:38.422 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 systemd[1]: Starting Ceph mds.d for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:38.708 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 podman[74324]: 2024-09-10 22:55:38.325331244 +0000 UTC m=+0.030051523 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:38.708 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 podman[74324]: 2024-09-10 22:55:38.435002765 +0000 UTC m=+0.139723043 container create e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T22:55:38.708 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 podman[74324]: 2024-09-10 22:55:38.662760803 +0000 UTC m=+0.367481083 container init e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T22:55:38.708 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 podman[74324]: 2024-09-10 22:55:38.669150614 +0000 UTC m=+0.373870901 container start e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T22:55:38.859 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:39.000 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 ceph-mds[74359]: -- 172.21.15.67:0/693479655 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4c086fc20 con 0x55e4c084c800 2024-09-10T22:55:39.000 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d[74355]: starting mds.d at 2024-09-10T22:55:39.000 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 bash[74324]: e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 2024-09-10T22:55:39.000 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 22:55:38 smithi067 systemd[1]: Started Ceph mds.d for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:39.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.466+0000 7f03bcc00640 1 -- 172.21.15.67:0/3399395066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b810b180 msgr2=0x7f03b8111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.466+0000 7f03bcc00640 1 --2- 172.21.15.67:0/3399395066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b810b180 0x7f03b8111980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009ca0 tx=0x7f03b002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 -- 172.21.15.67:0/3399395066 shutdown_connections 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 --2- 172.21.15.67:0/3399395066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b810b180 0x7f03b8111980 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 --2- 172.21.15.67:0/3399395066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810a800 0x7f03b810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 --2- 172.21.15.67:0/3399395066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b8104df0 0x7f03b81051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 -- 172.21.15.67:0/3399395066 >> 172.21.15.67:0/3399395066 conn(0x7f03b8100840 msgr2=0x7f03b8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 -- 172.21.15.67:0/3399395066 shutdown_connections 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 -- 172.21.15.67:0/3399395066 wait complete. 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.467+0000 7f03bcc00640 1 Processor -- start 2024-09-10T22:55:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 -- start start 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b810a800 0x7f03b819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810b180 0x7f03b81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8114d40 con 0x7f03b8104df0 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8114bc0 con 0x7f03b810a800 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03bcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8114ec0 con 0x7f03b810b180 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56894/0 (socket says 172.21.15.67:56894) 2024-09-10T22:55:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03b7400640 1 -- 172.21.15.67:0/1721160260 learned_addr learned my addr 172.21.15.67:0/1721160260 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.468+0000 7f03b7e00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810b180 0x7f03b81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03aea00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b810a800 0x7f03b819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 -- 172.21.15.67:0/1721160260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810b180 msgr2=0x7f03b81a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810b180 0x7f03b81a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 -- 172.21.15.67:0/1721160260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b810a800 msgr2=0x7f03b819b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b810a800 0x7f03b819b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 -- 172.21.15.67:0/1721160260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b81a10e0 con 0x7f03b8104df0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b7400640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c920 tx=0x7f03a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.469+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8007c70 con 0x7f03b8104df0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.470+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03a8007e10 con 0x7f03b8104df0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.470+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b81a13d0 con 0x7f03b8104df0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.470+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8018710 con 0x7f03b8104df0 2024-09-10T22:55:39.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.470+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b81a7a90 con 0x7f03b8104df0 2024-09-10T22:55:39.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.471+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b81051d0 con 0x7f03b8104df0 2024-09-10T22:55:39.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.473+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f03a800f450 con 0x7f03b8104df0 2024-09-10T22:55:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.476+0000 7f03b4c00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 0x7f039c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.476+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f03a80e5c90 con 0x7f03b8104df0 2024-09-10T22:55:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.476+0000 7f03aea00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 0x7f039c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.477+0000 7f03aea00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 0x7f039c0b78a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03b8103010 tx=0x7f03a000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.478+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03a8014030 con 0x7f03b8104df0 2024-09-10T22:55:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.711+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03b81a7e20 con 0x7f03b8104df0 2024-09-10T22:55:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.712+0000 7f03b4c00640 1 -- 172.21.15.67:0/1721160260 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3901 (secure 0 0 0) 0x7f03a80acf10 con 0x7f03b8104df0 2024-09-10T22:55:39.713 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:39.713 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"btime":"2024-09-10T22:55:39:099346+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:55:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2024-09-10T22:55:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.714+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 msgr2=0x7f039c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.714+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 0x7f039c0b78a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03b8103010 tx=0x7f03a000b390 comp rx=0 tx=0).stop 2024-09-10T22:55:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.714+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 msgr2=0x7f03b819b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.714+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c920 tx=0x7f03a800cde0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 shutdown_connections 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f039c0b53e0 0x7f039c0b78a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03b810b180 0x7f03b81a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03b810a800 0x7f03b819b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 --2- 172.21.15.67:0/1721160260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03b8104df0 0x7f03b819b030 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 >> 172.21.15.67:0/1721160260 conn(0x7f03b8100840 msgr2=0x7f03b8103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 shutdown_connections 2024-09-10T22:55:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:39.715+0000 7f03bcc00640 1 -- 172.21.15.67:0/1721160260 wait complete. 2024-09-10T22:55:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: Deploying daemon mds.e on smithi092 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.67:6836/1279733318,v1:172.21.15.67:6837/1279733318] up:boot 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: fsmap 4 up:standby 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:55:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:39 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1721160260' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: Deploying daemon mds.e on smithi092 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.67:6836/1279733318,v1:172.21.15.67:6837/1279733318] up:boot 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: fsmap 4 up:standby 2024-09-10T22:55:40.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:55:40.134 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:39 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1721160260' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: Deploying daemon mds.e on smithi092 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.67:6836/1279733318,v1:172.21.15.67:6837/1279733318] up:boot 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: fsmap 4 up:standby 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:55:40.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:39 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1721160260' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:41.639 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:41 smithi092 systemd[1]: Starting Ceph mds.e for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:41.942 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:41 smithi092 ceph-mon[34223]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:41.942 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:41 smithi092 podman[64840]: 2024-09-10 22:55:41.542894342 +0000 UTC m=+0.026897505 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:41.942 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:41 smithi092 podman[64840]: 2024-09-10 22:55:41.811362151 +0000 UTC m=+0.295365312 container create 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:55:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:41 smithi145 ceph-mon[34397]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:41 smithi067 ceph-mon[30939]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:42.521 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 podman[64840]: 2024-09-10 22:55:42.055695105 +0000 UTC m=+0.539698267 container init 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:55:42.522 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 podman[64840]: 2024-09-10 22:55:42.061984318 +0000 UTC m=+0.545987480 container start 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T22:55:42.522 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 ceph-mds[64861]: -- 172.21.15.92:0/2995054234 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6fc8d1860 con 0x55c6fc8ae800 2024-09-10T22:55:42.522 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e[64857]: starting mds.e at 2024-09-10T22:55:42.522 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 bash[64840]: 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 2024-09-10T22:55:42.522 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 22:55:42 smithi092 systemd[1]: Started Ceph mds.e for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:42.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:43.026 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:43.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: Deploying daemon mds.f on smithi145 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6834/2446174169,v1:172.21.15.92:6835/2446174169] up:boot 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: fsmap 5 up:standby 2024-09-10T22:55:43.502 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: Deploying daemon mds.f on smithi145 2024-09-10T22:55:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6834/2446174169,v1:172.21.15.92:6835/2446174169] up:boot 2024-09-10T22:55:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: fsmap 5 up:standby 2024-09-10T22:55:43.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:55:43.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 -- 172.21.15.67:0/3079941724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108fa0 msgr2=0x7f22cc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 --2- 172.21.15.67:0/3079941724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108fa0 0x7f22cc10f7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009ca0 tx=0x7f22c402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 -- 172.21.15.67:0/3079941724 shutdown_connections 2024-09-10T22:55:43.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 --2- 172.21.15.67:0/3079941724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108fa0 0x7f22cc10f7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 --2- 172.21.15.67:0/3079941724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108620 0x7f22cc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 --2- 172.21.15.67:0/3079941724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 -- 172.21.15.67:0/3079941724 >> 172.21.15.67:0/3079941724 conn(0x7f22cc0fe660 msgr2=0x7f22cc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 -- 172.21.15.67:0/3079941724 shutdown_connections 2024-09-10T22:55:43.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.614+0000 7f22d3000640 1 -- 172.21.15.67:0/3079941724 wait complete. 2024-09-10T22:55:43.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.615+0000 7f22d3000640 1 Processor -- start 2024-09-10T22:55:43.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 -- start start 2024-09-10T22:55:43.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 0x7f22cc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108fa0 0x7f22cc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc112b20 con 0x7f22cc108620 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1129a0 con 0x7f22cc102c10 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d3000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc112ca0 con 0x7f22cc108fa0 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 0x7f22cc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47708/0 (socket says 172.21.15.67:47708) 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.616+0000 7f22d1c00640 1 -- 172.21.15.67:0/2515378033 learned_addr learned my addr 172.21.15.67:0/2515378033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:43.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d2600640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108fa0 0x7f22cc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 -- 172.21.15.67:0/2515378033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108fa0 msgr2=0x7f22cc075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108fa0 0x7f22cc075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 -- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 msgr2=0x7f22cc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 0x7f22cc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 -- 172.21.15.67:0/2515378033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc1a7370 con 0x7f22cc102c10 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1200640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 0x7f22cc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22d1c00640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22c000ea60 tx=0x7f22c000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.617+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c000cde0 con 0x7f22cc102c10 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.618+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c00035f0 con 0x7f22cc102c10 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.618+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc1a7660 con 0x7f22cc102c10 2024-09-10T22:55:43.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.618+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0010690 con 0x7f22cc102c10 2024-09-10T22:55:43.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.618+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc1a7bf0 con 0x7f22cc102c10 2024-09-10T22:55:43.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.619+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc102ff0 con 0x7f22cc102c10 2024-09-10T22:55:43.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.621+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f22c0004000 con 0x7f22cc102c10 2024-09-10T22:55:43.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.623+0000 7f22ca000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 0x7f22b40b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.623+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f22c00e5e00 con 0x7f22cc102c10 2024-09-10T22:55:43.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.623+0000 7f22d1200640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 0x7f22b40b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.623+0000 7f22d1200640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 0x7f22b40b7920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22cc076f90 tx=0x7f22b8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.628+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22c0056220 con 0x7f22cc102c10 2024-09-10T22:55:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: Deploying daemon mds.f on smithi145 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6834/2446174169,v1:172.21.15.92:6835/2446174169] up:boot 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: fsmap 5 up:standby 2024-09-10T22:55:43.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:55:43.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.862+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22cc0767a0 con 0x7f22cc102c10 2024-09-10T22:55:43.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.863+0000 7f22ca000640 1 -- 172.21.15.67:0/2515378033 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4726 (secure 0 0 0) 0x7f22cc0767a0 con 0x7f22cc102c10 2024-09-10T22:55:43.864 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:43.865 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":6,"btime":"2024-09-10T22:55:42:863112+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:55:43.865 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 6 2024-09-10T22:55:43.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.865+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 msgr2=0x7f22b40b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.865+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 0x7f22b40b7920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22cc076f90 tx=0x7f22b8005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.865+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 msgr2=0x7f22cc0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.865+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22c000ea60 tx=0x7f22c000ef20 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 shutdown_connections 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22b40b5460 0x7f22b40b7920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22cc108fa0 0x7f22cc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22cc108620 0x7f22cc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 --2- 172.21.15.67:0/2515378033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22cc102c10 0x7f22cc0754a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 >> 172.21.15.67:0/2515378033 conn(0x7f22cc0fe660 msgr2=0x7f22cc100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 shutdown_connections 2024-09-10T22:55:43.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:43.866+0000 7f22d3000640 1 -- 172.21.15.67:0/2515378033 wait complete. 2024-09-10T22:55:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:44 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2515378033' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:44.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:44 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2515378033' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:44.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:44 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2515378033' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:45.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:45 smithi145 ceph-mon[34397]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:45 smithi067 ceph-mon[30939]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:45 smithi092 ceph-mon[34223]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:45.880 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 22:55:45 smithi145 systemd[1]: Started Ceph mds.f for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:46.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: Deploying daemon mds.c on smithi145 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6834/2757139462,v1:172.21.15.145:6835/2757139462] up:boot 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: fsmap 6 up:standby 2024-09-10T22:55:47.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:55:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:47.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:47.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: Deploying daemon mds.c on smithi145 2024-09-10T22:55:47.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6834/2757139462,v1:172.21.15.145:6835/2757139462] up:boot 2024-09-10T22:55:47.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: fsmap 6 up:standby 2024-09-10T22:55:47.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:55:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: Deploying daemon mds.c on smithi145 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6834/2757139462,v1:172.21.15.145:6835/2757139462] up:boot 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: fsmap 6 up:standby 2024-09-10T22:55:47.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:55:47.143 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3635934605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac075f40 msgr2=0x7f3bac10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/3635934605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac075f40 0x7f3bac10ee10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009900 tx=0x7f3b9402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3635934605 shutdown_connections 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/3635934605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/3635934605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac075f40 0x7f3bac10ee10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/3635934605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075540 0x7f3bac075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3635934605 >> 172.21.15.67:0/3635934605 conn(0x7f3bac0fe830 msgr2=0x7f3bac100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3635934605 shutdown_connections 2024-09-10T22:55:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.745+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3635934605 wait complete. 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.746+0000 7f3bb1a00640 1 Processor -- start 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.746+0000 7f3bb1a00640 1 -- start start 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac075540 0x7f3bac19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075f40 0x7f3bac19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac114e80 con 0x7f3bac10f3e0 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac114d00 con 0x7f3bac075540 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bb1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac115000 con 0x7f3bac075f40 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56934/0 (socket says 172.21.15.67:56934) 2024-09-10T22:55:47.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 -- 172.21.15.67:0/1387818383 learned_addr learned my addr 172.21.15.67:0/1387818383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:47.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3ba2000640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075f40 0x7f3bac19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:47.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3baaa00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac075540 0x7f3bac19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 -- 172.21.15.67:0/1387818383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075f40 msgr2=0x7f3bac19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075f40 0x7f3bac19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 -- 172.21.15.67:0/1387818383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac075540 msgr2=0x7f3bac19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac075540 0x7f3bac19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.747+0000 7f3bab400640 1 -- 172.21.15.67:0/1387818383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac1a1500 con 0x7f3bac10f3e0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3bab400640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400dac0 tx=0x7f3ba400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4017070 con 0x7f3bac10f3e0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba400be30 con 0x7f3bac10f3e0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac1a17f0 con 0x7f3bac10f3e0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4010460 con 0x7f3bac10f3e0 2024-09-10T22:55:47.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.748+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a7eb0 con 0x7f3bac10f3e0 2024-09-10T22:55:47.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.749+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac075920 con 0x7f3bac10f3e0 2024-09-10T22:55:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.751+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f3ba4003970 con 0x7f3bac10f3e0 2024-09-10T22:55:47.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.753+0000 7f3ba3e00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 0x7f3b900b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:47.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.753+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f3ba40e54b0 con 0x7f3bac10f3e0 2024-09-10T22:55:47.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.753+0000 7f3baaa00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 0x7f3b900b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:47.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.754+0000 7f3baaa00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 0x7f3b900b77f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bbe0 tx=0x7f3b9c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:47.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.757+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3ba40ac0f0 con 0x7f3bac10f3e0 2024-09-10T22:55:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:47 smithi145 ceph-mon[34397]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.991+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bac02e5c0 con 0x7f3bac10f3e0 2024-09-10T22:55:47.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:47 smithi067 ceph-mon[30939]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.991+0000 7f3ba3e00640 1 -- 172.21.15.67:0/1387818383 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5554 (secure 0 0 0) 0x7f3ba40b0b20 con 0x7f3bac10f3e0 2024-09-10T22:55:47.994 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:47.994 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":7,"btime":"2024-09-10T22:55:46:295505+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:55:47.994 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 7 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 msgr2=0x7f3b900b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 0x7f3b900b77f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bbe0 tx=0x7f3b9c00b390 comp rx=0 tx=0).stop 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 msgr2=0x7f3bac1a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400dac0 tx=0x7f3ba400df80 comp rx=0 tx=0).stop 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 shutdown_connections 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3b900b5330 0x7f3b900b77f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac10f3e0 0x7f3bac1a0d30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bac075f40 0x7f3bac19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1387818383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bac075540 0x7f3bac19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 >> 172.21.15.67:0/1387818383 conn(0x7f3bac0fe830 msgr2=0x7f3bac0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 shutdown_connections 2024-09-10T22:55:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:47.995+0000 7f3bb1a00640 1 -- 172.21.15.67:0/1387818383 wait complete. 2024-09-10T22:55:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:47 smithi092 ceph-mon[34223]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:48.488 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 systemd[1]: Starting Ceph mds.c for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:48.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1387818383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:48.790 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 podman[66365]: 2024-09-10 22:55:48.390997898 +0000 UTC m=+0.026945180 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:48.791 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 podman[66365]: 2024-09-10 22:55:48.512943498 +0000 UTC m=+0.148890795 container create 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T22:55:48.791 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 podman[66365]: 2024-09-10 22:55:48.784777297 +0000 UTC m=+0.420724558 container init 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:55:49.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1387818383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:49.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1387818383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:49.133 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 podman[66365]: 2024-09-10 22:55:48.791549427 +0000 UTC m=+0.427496688 container start 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T22:55:49.133 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 ceph-mds[66384]: -- 172.21.15.145:0/1212274362 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579a5387860 con 0x5579a5364800 2024-09-10T22:55:49.133 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: starting mds.c at 2024-09-10T22:55:49.133 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 bash[66365]: 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 2024-09-10T22:55:49.133 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 22:55:48 smithi145 systemd[1]: Started Ceph mds.c for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:49.939 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:49 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:49 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:50.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:50.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:50.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:49 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:50.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:50.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:50 smithi092 ceph-mon[34223]: Deploying daemon mds.h on smithi092 2024-09-10T22:55:50.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:50 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6836/1285378679,v1:172.21.15.145:6837/1285378679] up:boot 2024-09-10T22:55:50.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:50 smithi092 ceph-mon[34223]: fsmap 7 up:standby 2024-09-10T22:55:50.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:50 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:55:51.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:50 smithi067 ceph-mon[30939]: Deploying daemon mds.h on smithi092 2024-09-10T22:55:51.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:50 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6836/1285378679,v1:172.21.15.145:6837/1285378679] up:boot 2024-09-10T22:55:51.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:50 smithi067 ceph-mon[30939]: fsmap 7 up:standby 2024-09-10T22:55:51.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:50 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:55:51.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:50 smithi145 ceph-mon[34397]: Deploying daemon mds.h on smithi092 2024-09-10T22:55:51.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:50 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6836/1285378679,v1:172.21.15.145:6837/1285378679] up:boot 2024-09-10T22:55:51.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:50 smithi145 ceph-mon[34397]: fsmap 7 up:standby 2024-09-10T22:55:51.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:50 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:55:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 -- 172.21.15.67:0/1679824665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c01045b0 msgr2=0x7f01c010caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 --2- 172.21.15.67:0/1679824665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c01045b0 0x7f01c010caa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009900 tx=0x7f01ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 -- 172.21.15.67:0/1679824665 shutdown_connections 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 --2- 172.21.15.67:0/1679824665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c010d0f0 0x7f01c010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 --2- 172.21.15.67:0/1679824665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c01045b0 0x7f01c010caa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 --2- 172.21.15.67:0/1679824665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c0103c90 0x7f01c0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 -- 172.21.15.67:0/1679824665 >> 172.21.15.67:0/1679824665 conn(0x7f01c00fd710 msgr2=0x7f01c00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 -- 172.21.15.67:0/1679824665 shutdown_connections 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.829+0000 7f01c8600640 1 -- 172.21.15.67:0/1679824665 wait complete. 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 Processor -- start 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 -- start start 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c0103c90 0x7f01c01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c01045b0 0x7f01c01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0112c10 con 0x7f01c010d0f0 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0112a90 con 0x7f01c01045b0 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.830+0000 7f01c8600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0112d90 con 0x7f01c0103c90 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51738/0 (socket says 172.21.15.67:51738) 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 -- 172.21.15.67:0/3687826673 learned_addr learned my addr 172.21.15.67:0/3687826673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c5c00640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c0103c90 0x7f01c01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c5200640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c01045b0 0x7f01c01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 -- 172.21.15.67:0/3687826673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c0103c90 msgr2=0x7f01c01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c0103c90 0x7f01c01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 -- 172.21.15.67:0/3687826673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c01045b0 msgr2=0x7f01c01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c01045b0 0x7f01c01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.831+0000 7f01c6600640 1 -- 172.21.15.67:0/3687826673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c019f250 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01c6600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01b800eaa0 tx=0x7f01b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b800cdb0 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b8005020 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80116d0 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c019f540 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.832+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a5c00 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.833+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c0104070 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.836+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f01b800f040 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.837+0000 7f01be000640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 0x7f01a80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.837+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f01b80e5680 con 0x7f01c010d0f0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.837+0000 7f01c5c00640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 0x7f01a80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.838+0000 7f01c5c00640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 0x7f01a80b77f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009bd0 tx=0x7f01b4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:51.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:51.841+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01b8016030 con 0x7f01c010d0f0 2024-09-10T22:55:51.905 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:51 smithi092 ceph-mon[34223]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:51 smithi067 ceph-mon[30939]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.073+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01c002e5c0 con 0x7f01c010d0f0 2024-09-10T22:55:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.074+0000 7f01be000640 1 -- 172.21.15.67:0/3687826673 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6382 (secure 0 0 0) 0x7f01b80ac220 con 0x7f01c010d0f0 2024-09-10T22:55:52.076 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"btime":"2024-09-10T22:55:49:693953+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T22:55:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-09-10T22:55:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 msgr2=0x7f01a80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 0x7f01a80b77f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009bd0 tx=0x7f01b4005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 msgr2=0x7f01c019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01b800eaa0 tx=0x7f01b800ef60 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 shutdown_connections 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01a80b5330 0x7f01a80b77f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c010d0f0 0x7f01c019ea80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01c01045b0 0x7f01c01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.076+0000 7f01c8600640 1 --2- 172.21.15.67:0/3687826673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01c0103c90 0x7f01c01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.077+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 >> 172.21.15.67:0/3687826673 conn(0x7f01c00fd710 msgr2=0x7f01c00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.077+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 shutdown_connections 2024-09-10T22:55:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:52.077+0000 7f01c8600640 1 -- 172.21.15.67:0/3687826673 wait complete. 2024-09-10T22:55:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:51 smithi145 ceph-mon[34397]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:52.159 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:51 smithi092 podman[65154]: 2024-09-10 22:55:51.921748253 +0000 UTC m=+0.140149609 container create 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T22:55:52.159 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 podman[65154]: 2024-09-10 22:55:52.157928669 +0000 UTC m=+0.376330028 container init 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T22:55:52.521 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 podman[65154]: 2024-09-10 22:55:52.164070095 +0000 UTC m=+0.382471454 container start 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T22:55:52.522 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mds[65173]: -- 172.21.15.92:0/3189590483 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e141f3860 con 0x564e141d1000 2024-09-10T22:55:52.522 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: starting mds.h at 2024-09-10T22:55:52.522 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 bash[65154]: 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 2024-09-10T22:55:52.522 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 22:55:52 smithi092 systemd[1]: Started Ceph mds.h for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3687826673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:52.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:53.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3687826673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:53.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3687826673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:53.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:53 smithi092 ceph-mon[34223]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:53 smithi092 ceph-mon[34223]: Deploying daemon mds.i on smithi145 2024-09-10T22:55:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:53 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:boot 2024-09-10T22:55:54.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:53 smithi092 ceph-mon[34223]: fsmap 8 up:standby 2024-09-10T22:55:54.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:55:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:53 smithi067 ceph-mon[30939]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:54.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:53 smithi067 ceph-mon[30939]: Deploying daemon mds.i on smithi145 2024-09-10T22:55:54.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:53 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:boot 2024-09-10T22:55:54.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:53 smithi067 ceph-mon[30939]: fsmap 8 up:standby 2024-09-10T22:55:54.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:55:54.078 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:53 smithi145 ceph-mon[34397]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:54.078 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:53 smithi145 ceph-mon[34397]: Deploying daemon mds.i on smithi145 2024-09-10T22:55:54.079 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:53 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:boot 2024-09-10T22:55:54.079 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:53 smithi145 ceph-mon[34397]: fsmap 8 up:standby 2024-09-10T22:55:54.079 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:55:55.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:55.130 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:54 smithi145 systemd[1]: Starting Ceph mds.i for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:55.441 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 podman[66664]: 2024-09-10 22:55:55.058143068 +0000 UTC m=+0.027450481 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:55.441 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 podman[66664]: 2024-09-10 22:55:55.170537241 +0000 UTC m=+0.139844653 container create 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T22:55:55.441 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 podman[66664]: 2024-09-10 22:55:55.440269623 +0000 UTC m=+0.409577057 container init 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T22:55:55.726 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 podman[66664]: 2024-09-10 22:55:55.446549584 +0000 UTC m=+0.415857011 container start 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T22:55:55.726 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mds[66683]: -- 172.21.15.145:0/3419145644 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625d85f5860 con 0x5625d85d3000 2024-09-10T22:55:55.726 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: starting mds.i at 2024-09-10T22:55:55.726 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 bash[66664]: 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 2024-09-10T22:55:55.726 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 22:55:55 smithi145 systemd[1]: Started Ceph mds.i for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:55:55.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:55.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:55.817 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:55.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:55.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:55.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:55.818 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:56.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:56.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 -- 172.21.15.67:0/1257706184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 msgr2=0x7f251c10ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 --2- 172.21.15.67:0/1257706184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c10ecc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f250c009900 tx=0x7f250c02a410 comp rx=0 tx=0).stop 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 -- 172.21.15.67:0/1257706184 shutdown_connections 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 --2- 172.21.15.67:0/1257706184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 0x7f251c1115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 --2- 172.21.15.67:0/1257706184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c10ecc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.059+0000 7f2523200640 1 --2- 172.21.15.67:0/1257706184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 0x7f251c0758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.060+0000 7f2523200640 1 -- 172.21.15.67:0/1257706184 >> 172.21.15.67:0/1257706184 conn(0x7f251c0fe660 msgr2=0x7f251c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.060+0000 7f2523200640 1 -- 172.21.15.67:0/1257706184 shutdown_connections 2024-09-10T22:55:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.060+0000 7f2523200640 1 -- 172.21.15.67:0/1257706184 wait complete. 2024-09-10T22:55:56.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.061+0000 7f2523200640 1 Processor -- start 2024-09-10T22:55:56.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.061+0000 7f2523200640 1 -- start start 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.061+0000 7f2523200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 0x7f251c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.061+0000 7f2523200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.061+0000 7f2523200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 0x7f251c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2523200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f251c114ce0 con 0x7f251c075e00 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2523200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f251c114b60 con 0x7f251c0754e0 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2523200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f251c114e60 con 0x7f251c10f200 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51770/0 (socket says 172.21.15.67:51770) 2024-09-10T22:55:56.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 -- 172.21.15.67:0/115805014 learned_addr learned my addr 172.21.15.67:0/115805014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2522800640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 0x7f251c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521e00640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 0x7f251c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 -- 172.21.15.67:0/115805014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 msgr2=0x7f251c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 0x7f251c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 -- 172.21.15.67:0/115805014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 msgr2=0x7f251c19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 0x7f251c19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.062+0000 7f2521400640 1 -- 172.21.15.67:0/115805014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a1240 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2521400640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f250c02a930 tx=0x7f250c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c03f070 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f250c003e40 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a14d0 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c035030 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.063+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a7bf0 con 0x7f251c075e00 2024-09-10T22:55:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.064+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251c0758c0 con 0x7f251c075e00 2024-09-10T22:55:56.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.066+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f250c031400 con 0x7f251c075e00 2024-09-10T22:55:56.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.068+0000 7f2516a00640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 0x7f25040b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:56.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.068+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f250c1064a0 con 0x7f251c075e00 2024-09-10T22:55:56.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.068+0000 7f2521e00640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 0x7f25040b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:56.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.069+0000 7f2521e00640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 0x7f25040b78a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2510005d00 tx=0x7f2510005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:56.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.072+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f250c0cd690 con 0x7f251c075e00 2024-09-10T22:55:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:55:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:55:56.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:55:56.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:56.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.319+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f251c1a7ee0 con 0x7f251c075e00 2024-09-10T22:55:56.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.320+0000 7f2516a00640 1 -- 172.21.15.67:0/115805014 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8034 (secure 0 0 0) 0x7f250c0ccf90 con 0x7f251c075e00 2024-09-10T22:55:56.322 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:55:56.322 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"btime":"2024-09-10T22:55:55:722174+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:55:56.322 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 10 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 msgr2=0x7f25040b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 0x7f25040b78a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2510005d00 tx=0x7f2510005c50 comp rx=0 tx=0).stop 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 msgr2=0x7f251c19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f250c02a930 tx=0x7f250c003c20 comp rx=0 tx=0).stop 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 shutdown_connections 2024-09-10T22:55:56.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25040b53e0 0x7f25040b78a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f251c10f200 0x7f251c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f251c075e00 0x7f251c19b6d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 --2- 172.21.15.67:0/115805014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f251c0754e0 0x7f251c19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.322+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 >> 172.21.15.67:0/115805014 conn(0x7f251c0fe660 msgr2=0x7f251c111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.323+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 shutdown_connections 2024-09-10T22:55:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:55:56.323+0000 7f2523200640 1 -- 172.21.15.67:0/115805014 wait complete. 2024-09-10T22:55:57.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:56 smithi092 ceph-mon[34223]: Deploying daemon mds.g on smithi067 2024-09-10T22:55:57.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:56 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6838/4174586003,v1:172.21.15.145:6839/4174586003] up:boot 2024-09-10T22:55:57.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:56 smithi092 ceph-mon[34223]: fsmap 9 up:standby 2024-09-10T22:55:57.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:55:57.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:56 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/115805014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:57.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:56 smithi067 ceph-mon[30939]: Deploying daemon mds.g on smithi067 2024-09-10T22:55:57.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:56 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6838/4174586003,v1:172.21.15.145:6839/4174586003] up:boot 2024-09-10T22:55:57.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:56 smithi067 ceph-mon[30939]: fsmap 9 up:standby 2024-09-10T22:55:57.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:55:57.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:56 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/115805014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:56 smithi145 ceph-mon[34397]: Deploying daemon mds.g on smithi067 2024-09-10T22:55:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:56 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6838/4174586003,v1:172.21.15.145:6839/4174586003] up:boot 2024-09-10T22:55:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:56 smithi145 ceph-mon[34397]: fsmap 9 up:standby 2024-09-10T22:55:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:55:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:56 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/115805014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:58.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:57 smithi067 ceph-mon[30939]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:57 smithi092 ceph-mon[34223]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:57 smithi145 ceph-mon[34397]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:55:59.386 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 systemd[1]: Starting Ceph mds.g for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:55:59.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:59.742 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 podman[75159]: 2024-09-10 22:55:59.289401215 +0000 UTC m=+0.027280719 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:55:59.743 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 podman[75159]: 2024-09-10 22:55:59.454477763 +0000 UTC m=+0.192357247 container create 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T22:55:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:55:59 smithi092 ceph-mon[34223]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:55:59 smithi067 ceph-mon[30939]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:00.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 podman[75159]: 2024-09-10 22:55:59.749175206 +0000 UTC m=+0.487054707 container init 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T22:56:00.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 podman[75159]: 2024-09-10 22:55:59.756041385 +0000 UTC m=+0.493920864 container start 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:56:00.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 ceph-mds[75207]: -- 172.21.15.67:0/3732804682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55616117d860 con 0x55616115b400 2024-09-10T22:56:00.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g[75192]: starting mds.g at 2024-09-10T22:56:00.049 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 bash[75159]: 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a 2024-09-10T22:56:00.049 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 22:55:59 smithi067 systemd[1]: Started Ceph mds.g for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:56:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:55:59 smithi145 ceph-mon[34397]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:00.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 -- 172.21.15.67:0/4023248797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4113980 msgr2=0x7efee4115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 --2- 172.21.15.67:0/4023248797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4113980 0x7efee4115d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efedc009ca0 tx=0x7efedc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 -- 172.21.15.67:0/4023248797 shutdown_connections 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 --2- 172.21.15.67:0/4023248797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4113980 0x7efee4115d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 --2- 172.21.15.67:0/4023248797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4077ff0 0x7efee4113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 --2- 172.21.15.67:0/4023248797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee4077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 -- 172.21.15.67:0/4023248797 >> 172.21.15.67:0/4023248797 conn(0x7efee4100a00 msgr2=0x7efee4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 -- 172.21.15.67:0/4023248797 shutdown_connections 2024-09-10T22:56:00.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.660+0000 7efeec200640 1 -- 172.21.15.67:0/4023248797 wait complete. 2024-09-10T22:56:00.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 Processor -- start 2024-09-10T22:56:00.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 -- start start 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 0x7efee419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4113980 0x7efee41a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efee4119250 con 0x7efee4077ff0 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efee41190d0 con 0x7efee40776d0 2024-09-10T22:56:00.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.661+0000 7efeec200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efee41193d0 con 0x7efee4113980 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 0x7efee419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:42720/0 (socket says 172.21.15.67:42720) 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 -- 172.21.15.67:0/298679755 learned_addr learned my addr 172.21.15.67:0/298679755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efeea200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4113980 0x7efee41a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 -- 172.21.15.67:0/298679755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4113980 msgr2=0x7efee41a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4113980 0x7efee41a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 -- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 msgr2=0x7efee419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 0x7efee419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee9800640 1 -- 172.21.15.67:0/298679755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee41a5810 con 0x7efee40776d0 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.662+0000 7efee8e00640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 0x7efee419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efee9800640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efed800ea60 tx=0x7efed800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed800cde0 con 0x7efee40776d0 2024-09-10T22:56:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed80035f0 con 0x7efee40776d0 2024-09-10T22:56:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8010690 con 0x7efee40776d0 2024-09-10T22:56:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee41a5b00 con 0x7efee40776d0 2024-09-10T22:56:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.663+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41ac390 con 0x7efee40776d0 2024-09-10T22:56:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.665+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee4077ab0 con 0x7efee40776d0 2024-09-10T22:56:00.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.667+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7efed8004000 con 0x7efee40776d0 2024-09-10T22:56:00.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.668+0000 7efee2000640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 0x7efecc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:00.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.669+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7efed80e5c00 con 0x7efee40776d0 2024-09-10T22:56:00.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.669+0000 7efee8e00640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 0x7efecc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:00.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.669+0000 7efee8e00640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 0x7efecc0b7870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efee4115e50 tx=0x7efed000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:00.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.670+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efed80acdf0 con 0x7efee40776d0 2024-09-10T22:56:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.940+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee41a0890 con 0x7efee40776d0 2024-09-10T22:56:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.942+0000 7efee2000640 1 -- 172.21.15.67:0/298679755 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8860 (secure 0 0 0) 0x7efed80ac6f0 con 0x7efee40776d0 2024-09-10T22:56:00.944 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:00.944 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"btime":"2024-09-10T22:56:00:741570+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:56:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.944+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 msgr2=0x7efecc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.944+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 0x7efecc0b7870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efee4115e50 tx=0x7efed000b390 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 msgr2=0x7efee419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efed800ea60 tx=0x7efed800ef20 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 shutdown_connections 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efecc0b53b0 0x7efecc0b7870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efee4113980 0x7efee41a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efee4077ff0 0x7efee419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 --2- 172.21.15.67:0/298679755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efee40776d0 0x7efee419f760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 >> 172.21.15.67:0/298679755 conn(0x7efee4100a00 msgr2=0x7efee4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 shutdown_connections 2024-09-10T22:56:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:00.945+0000 7efeec200640 1 -- 172.21.15.67:0/298679755 wait complete. 2024-09-10T22:56:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.67:6838/4128597236,v1:172.21.15.67:6839/4128597236] up:boot 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: fsmap 10 up:standby 2024-09-10T22:56:01.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:56:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.67:6838/4128597236,v1:172.21.15.67:6839/4128597236] up:boot 2024-09-10T22:56:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: fsmap 10 up:standby 2024-09-10T22:56:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:56:01.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.67:6838/4128597236,v1:172.21.15.67:6839/4128597236] up:boot 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: fsmap 10 up:standby 2024-09-10T22:56:01.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:56:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:01 smithi067 ceph-mon[30939]: Deploying daemon mds.a on smithi067 2024-09-10T22:56:02.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:01 smithi067 ceph-mon[30939]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:02.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:01 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/298679755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:01 smithi092 ceph-mon[34223]: Deploying daemon mds.a on smithi067 2024-09-10T22:56:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:01 smithi092 ceph-mon[34223]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:02.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:01 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/298679755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:01 smithi145 ceph-mon[34397]: Deploying daemon mds.a on smithi067 2024-09-10T22:56:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:01 smithi145 ceph-mon[34397]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:02.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:01 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/298679755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:03.799 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:03 smithi067 systemd[1]: Starting Ceph mds.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:56:03.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:04.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:03 smithi067 ceph-mon[30939]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:04.135 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:03 smithi067 podman[75566]: 2024-09-10 22:56:03.73352758 +0000 UTC m=+0.027139579 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:56:04.135 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:03 smithi067 podman[75566]: 2024-09-10 22:56:03.847302223 +0000 UTC m=+0.140914225 container create e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T22:56:04.135 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 podman[75566]: 2024-09-10 22:56:04.133544571 +0000 UTC m=+0.427156596 container init e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T22:56:04.209 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:03 smithi145 ceph-mon[34397]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:03 smithi092 ceph-mon[34223]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:04.401 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:04.540 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 podman[75566]: 2024-09-10 22:56:04.140260842 +0000 UTC m=+0.433872851 container start e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:56:04.540 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 ceph-mds[75603]: -- 172.21.15.67:0/3329223685 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556474f2b860 con 0x556474f09400 2024-09-10T22:56:04.540 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a[75599]: starting mds.a at 2024-09-10T22:56:04.540 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 bash[75566]: e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b 2024-09-10T22:56:04.540 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 22:56:04 smithi067 systemd[1]: Started Ceph mds.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 -- 172.21.15.67:0/1758780639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108fa0 msgr2=0x7f758c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 --2- 172.21.15.67:0/1758780639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108fa0 0x7f758c10f7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7584009ca0 tx=0x7f758402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 -- 172.21.15.67:0/1758780639 shutdown_connections 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 --2- 172.21.15.67:0/1758780639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108fa0 0x7f758c10f7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 --2- 172.21.15.67:0/1758780639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c108620 0x7f758c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 --2- 172.21.15.67:0/1758780639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c102c10 0x7f758c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 -- 172.21.15.67:0/1758780639 >> 172.21.15.67:0/1758780639 conn(0x7f758c0fe660 msgr2=0x7f758c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 -- 172.21.15.67:0/1758780639 shutdown_connections 2024-09-10T22:56:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.981+0000 7f7590e00640 1 -- 172.21.15.67:0/1758780639 wait complete. 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 Processor -- start 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 -- start start 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108620 0x7f758c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c108fa0 0x7f758c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f758c112b60 con 0x7f758c108620 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1129e0 con 0x7f758c108fa0 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f7590e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f758c112ce0 con 0x7f758c102c10 2024-09-10T22:56:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f758b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f758aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108620 0x7f758c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f758b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40052/0 (socket says 172.21.15.67:40052) 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.983+0000 7f758b400640 1 -- 172.21.15.67:0/2711264956 learned_addr learned my addr 172.21.15.67:0/2711264956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758be00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c108fa0 0x7f758c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 -- 172.21.15.67:0/2711264956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c108fa0 msgr2=0x7f758c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c108fa0 0x7f758c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 -- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108620 msgr2=0x7f758c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108620 0x7f758c19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 -- 172.21.15.67:0/2711264956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1a11e0 con 0x7f758c102c10 2024-09-10T22:56:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.984+0000 7f758b400640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.985+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cd80 con 0x7f758c102c10 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.985+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f757c0108c0 con 0x7f758c102c10 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.985+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0187c0 con 0x7f758c102c10 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.985+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a14d0 con 0x7f758c102c10 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.985+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a7b90 con 0x7f758c102c10 2024-09-10T22:56:04.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.986+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c102ff0 con 0x7f758c102c10 2024-09-10T22:56:04.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.988+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f757c010430 con 0x7f758c102c10 2024-09-10T22:56:04.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.989+0000 7f7583e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 0x7f75700b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:04.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.990+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f757c0e5c00 con 0x7f758c102c10 2024-09-10T22:56:04.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.990+0000 7f758aa00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 0x7f75700b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:04.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.990+0000 7f758aa00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 0x7f75700b7920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f758c107ea0 tx=0x7f7578005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:04.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:04.991+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f757c0ac7a0 con 0x7f758c102c10 2024-09-10T22:56:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.224+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f758c19c260 con 0x7f758c102c10 2024-09-10T22:56:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.225+0000 7f7583e00640 1 -- 172.21.15.67:0/2711264956 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9686 (secure 0 0 0) 0x7f757c0b11d0 con 0x7f758c102c10 2024-09-10T22:56:05.227 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:05.227 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":12,"btime":"2024-09-10T22:56:04:944209+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:56:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 12 2024-09-10T22:56:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 msgr2=0x7f75700b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 0x7f75700b7920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f758c107ea0 tx=0x7f7578005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 msgr2=0x7f758c19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 shutdown_connections 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f75700b5460 0x7f75700b7920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f758c108fa0 0x7f758c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f758c108620 0x7f758c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 --2- 172.21.15.67:0/2711264956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f758c102c10 0x7f758c19b130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 >> 172.21.15.67:0/2711264956 conn(0x7f758c0fe660 msgr2=0x7f758c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 shutdown_connections 2024-09-10T22:56:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:05.227+0000 7f7590e00640 1 -- 172.21.15.67:0/2711264956 wait complete. 2024-09-10T22:56:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: Deploying daemon mds.b on smithi092 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.67:6840/2410494517,v1:172.21.15.67:6841/2410494517] up:boot 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: fsmap 11 up:standby 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:56:05.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:05 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2711264956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: Deploying daemon mds.b on smithi092 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.67:6840/2410494517,v1:172.21.15.67:6841/2410494517] up:boot 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: fsmap 11 up:standby 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:56:05.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:05 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2711264956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: Deploying daemon mds.b on smithi092 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.67:6840/2410494517,v1:172.21.15.67:6841/2410494517] up:boot 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: fsmap 11 up:standby 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:56:05.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:05 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2711264956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:07.185 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:06 smithi092 systemd[1]: Starting Ceph mds.b for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T22:56:07.468 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:07 smithi067 ceph-mon[30939]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:07.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:07 smithi092 ceph-mon[34223]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:07.496 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 podman[65454]: 2024-09-10 22:56:07.088365454 +0000 UTC m=+0.027076597 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:56:07.496 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 podman[65454]: 2024-09-10 22:56:07.217072752 +0000 UTC m=+0.155783894 container create dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:56:07.496 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 podman[65454]: 2024-09-10 22:56:07.49496291 +0000 UTC m=+0.433674069 container init dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T22:56:07.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:07 smithi145 ceph-mon[34397]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:07.771 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 podman[65454]: 2024-09-10 22:56:07.501489332 +0000 UTC m=+0.440200475 container start dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:56:07.771 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 ceph-mds[65474]: -- 172.21.15.92:0/2678452401 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c81989b860 con 0x55c819878800 2024-09-10T22:56:07.771 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: starting mds.b at 2024-09-10T22:56:07.772 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 bash[65454]: dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 2024-09-10T22:56:07.772 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:07 smithi092 systemd[1]: Started Ceph mds.b for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T22:56:08.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:08.537 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:boot 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: fsmap 12 up:standby 2024-09-10T22:56:09.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:boot 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: fsmap 12 up:standby 2024-09-10T22:56:09.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:56:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:boot 2024-09-10T22:56:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: fsmap 12 up:standby 2024-09-10T22:56:09.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 -- 172.21.15.67:0/3319237212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac8109150 msgr2=0x7f2ac810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 --2- 172.21.15.67:0/3319237212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac8109150 0x7f2ac810f950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009ca0 tx=0x7f2ac002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 -- 172.21.15.67:0/3319237212 shutdown_connections 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 --2- 172.21.15.67:0/3319237212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac8109150 0x7f2ac810f950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 --2- 172.21.15.67:0/3319237212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac81087d0 0x7f2ac8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 --2- 172.21.15.67:0/3319237212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8102dc0 0x7f2ac81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 -- 172.21.15.67:0/3319237212 >> 172.21.15.67:0/3319237212 conn(0x7f2ac80fe830 msgr2=0x7f2ac8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 -- 172.21.15.67:0/3319237212 shutdown_connections 2024-09-10T22:56:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.350+0000 7f2acde00640 1 -- 172.21.15.67:0/3319237212 wait complete. 2024-09-10T22:56:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.351+0000 7f2acde00640 1 Processor -- start 2024-09-10T22:56:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.351+0000 7f2acde00640 1 -- start start 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 0x7f2ac80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 0x7f2ac8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8112cd0 con 0x7f2ac81087d0 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8112b50 con 0x7f2ac8109150 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2acde00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8112e50 con 0x7f2ac8102dc0 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2ac7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 0x7f2ac8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2ac6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 0x7f2ac80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38536/0 (socket says 172.21.15.67:38536) 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.352+0000 7f2abfe00640 1 -- 172.21.15.67:0/4230889038 learned_addr learned my addr 172.21.15.67:0/4230889038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 -- 172.21.15.67:0/4230889038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 msgr2=0x7f2ac8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 0x7f2ac8075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 -- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 msgr2=0x7f2ac80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 -- 172.21.15.67:0/4230889038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac81a75b0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac7400640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 0x7f2ac8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2abfe00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2ac6a00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 0x7f2ac80754a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e9f0 tx=0x7f2ab800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800ccc0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.353+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac81a78a0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.354+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ab80108c0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.354+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80187c0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.354+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac81a7e30 con 0x7f2ac8102dc0 2024-09-10T22:56:09.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.355+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac81031a0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.357+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2ab8010430 con 0x7f2ac8102dc0 2024-09-10T22:56:09.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.358+0000 7f2acca00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 0x7f2aac0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:09.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.359+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f2ab80e5fe0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.359+0000 7f2abfe00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 0x7f2aac0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:09.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.360+0000 7f2abfe00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 0x7f2aac0b7870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8076f90 tx=0x7f2ab0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:09.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.360+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2ab80acba0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.595+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ac80767a0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.596+0000 7f2acca00640 1 -- 172.21.15.67:0/4230889038 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10512 (secure 0 0 0) 0x7f2ab80b15d0 con 0x7f2ac8102dc0 2024-09-10T22:56:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:09.599 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"btime":"2024-09-10T22:56:08:362956+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T22:56:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2024-09-10T22:56:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 msgr2=0x7f2aac0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 0x7f2aac0b7870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8076f90 tx=0x7f2ab0005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 msgr2=0x7f2ac80754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 0x7f2ac80754a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800e9f0 tx=0x7f2ab800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 shutdown_connections 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2aac0b53b0 0x7f2aac0b7870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ac8109150 0x7f2ac8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac81087d0 0x7f2ac80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 --2- 172.21.15.67:0/4230889038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ac8102dc0 0x7f2ac80754a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 >> 172.21.15.67:0/4230889038 conn(0x7f2ac80fe830 msgr2=0x7f2ac8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.599+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 shutdown_connections 2024-09-10T22:56:09.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:09.600+0000 7f2acde00640 1 -- 172.21.15.67:0/4230889038 wait complete. 2024-09-10T22:56:09.945 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:09 smithi067 ceph-mon[30939]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:09.945 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:09 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4230889038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:09 smithi092 ceph-mon[34223]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:09 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4230889038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:09 smithi145 ceph-mon[34397]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:09 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4230889038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:10.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T22:56:11.215 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:11.353 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:11 smithi067 ceph-mon[30939]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:11.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.354 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:11 smithi145 ceph-mon[34397]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:11 smithi092 ceph-mon[34223]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:11.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:11.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 -- 172.21.15.67:0/1719571733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010b320 msgr2=0x7fcab0111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 --2- 172.21.15.67:0/1719571733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010b320 0x7fcab0111b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009ca0 tx=0x7fcaa802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 -- 172.21.15.67:0/1719571733 shutdown_connections 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 --2- 172.21.15.67:0/1719571733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010b320 0x7fcab0111b20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 --2- 172.21.15.67:0/1719571733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab010a9a0 0x7fcab010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 --2- 172.21.15.67:0/1719571733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab0104f90 0x7fcab0105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 -- 172.21.15.67:0/1719571733 >> 172.21.15.67:0/1719571733 conn(0x7fcab0100a00 msgr2=0x7fcab0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.786+0000 7fcab8200640 1 -- 172.21.15.67:0/1719571733 shutdown_connections 2024-09-10T22:56:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.787+0000 7fcab8200640 1 -- 172.21.15.67:0/1719571733 wait complete. 2024-09-10T22:56:11.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.787+0000 7fcab8200640 1 Processor -- start 2024-09-10T22:56:11.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 -- start start 2024-09-10T22:56:11.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0104f90 0x7fcab019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab010b320 0x7fcab01a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0114ea0 con 0x7fcab010a9a0 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0114d20 con 0x7fcab0104f90 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0115020 con 0x7fcab010b320 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38558/0 (socket says 172.21.15.67:38558) 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab4e00640 1 -- 172.21.15.67:0/3425947954 learned_addr learned my addr 172.21.15.67:0/3425947954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab6200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab010b320 0x7fcab01a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.788+0000 7fcab5800640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0104f90 0x7fcab019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 -- 172.21.15.67:0/3425947954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab010b320 msgr2=0x7fcab01a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab010b320 0x7fcab01a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 -- 172.21.15.67:0/3425947954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0104f90 msgr2=0x7fcab019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0104f90 0x7fcab019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 -- 172.21.15.67:0/3425947954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab01a1460 con 0x7fcab010a9a0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab4e00640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b4a0 tx=0x7fca9c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c014070 con 0x7fcab010a9a0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca9c010aa0 con 0x7fcab010a9a0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c017b70 con 0x7fcab010a9a0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab01a1750 con 0x7fcab010a9a0 2024-09-10T22:56:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.789+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab01a7e10 con 0x7fcab010a9a0 2024-09-10T22:56:11.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.791+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab0105370 con 0x7fcab010a9a0 2024-09-10T22:56:11.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.793+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fca9c010610 con 0x7fcab010a9a0 2024-09-10T22:56:11.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.795+0000 7fcaae000640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 0x7fca980b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.795+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fca9c0e50c0 con 0x7fcab010a9a0 2024-09-10T22:56:11.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.795+0000 7fcab5800640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 0x7fca980b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.796+0000 7fcab5800640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 0x7fca980b7870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400bbe0 tx=0x7fcaa400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:11.799+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fca9c0abc60 con 0x7fcab010a9a0 2024-09-10T22:56:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.010+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcab002e5c0 con 0x7fcab010a9a0 2024-09-10T22:56:12.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.010+0000 7fcaae000640 1 -- 172.21.15.67:0/3425947954 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fca9c0b0690 con 0x7fcab010a9a0 2024-09-10T22:56:12.012 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:12.012 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-09-10T22:56:12.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T22:56:12.013 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-09-10T22:56:12.013 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-09-10T22:56:12.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T22:56:12.013 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-09-10T22:56:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 msgr2=0x7fca980b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 0x7fca980b7870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400bbe0 tx=0x7fcaa400b390 comp rx=0 tx=0).stop 2024-09-10T22:56:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 msgr2=0x7fcab019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b4a0 tx=0x7fca9c00b960 comp rx=0 tx=0).stop 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 shutdown_connections 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca980b53b0 0x7fca980b7870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab010b320 0x7fcab01a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab010a9a0 0x7fcab019b8f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 --2- 172.21.15.67:0/3425947954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0104f90 0x7fcab019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.013+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 >> 172.21.15.67:0/3425947954 conn(0x7fcab0100a00 msgr2=0x7fcab0102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.014+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 shutdown_connections 2024-09-10T22:56:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:12.014+0000 7fcab8200640 1 -- 172.21.15.67:0/3425947954 wait complete. 2024-09-10T22:56:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:12.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:12.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:12.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3425947954' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:56:12.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3425947954' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:56:12.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:12.926 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:12.927 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.927 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:12.927 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3425947954' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:56:12.927 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:12.961 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:56:12.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:56:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:13 smithi067 ceph-mon[30939]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:13 smithi145 ceph-mon[34397]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:13.938 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:13 smithi092 ceph-mon[34223]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:14.129 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 -- 172.21.15.67:0/1270931189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec102c00 msgr2=0x7f2aec102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 --2- 172.21.15.67:0/1270931189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec102c00 0x7f2aec102fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009ca0 tx=0x7f2adc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 -- 172.21.15.67:0/1270931189 shutdown_connections 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 --2- 172.21.15.67:0/1270931189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec108f90 0x7f2aec10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 --2- 172.21.15.67:0/1270931189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108610 0x7f2aec108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 --2- 172.21.15.67:0/1270931189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec102c00 0x7f2aec102fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.700+0000 7f2aeb200640 1 -- 172.21.15.67:0/1270931189 >> 172.21.15.67:0/1270931189 conn(0x7f2aec0fe6d0 msgr2=0x7f2aec100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.701+0000 7f2aeb200640 1 -- 172.21.15.67:0/1270931189 shutdown_connections 2024-09-10T22:56:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.701+0000 7f2aeb200640 1 -- 172.21.15.67:0/1270931189 wait complete. 2024-09-10T22:56:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.701+0000 7f2aeb200640 1 Processor -- start 2024-09-10T22:56:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 -- start start 2024-09-10T22:56:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec108610 0x7f2aec19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108f90 0x7f2aec1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec112b50 con 0x7f2aec108610 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1129d0 con 0x7f2aec102c00 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.702+0000 7f2aeb200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec112cd0 con 0x7f2aec108f90 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec108610 0x7f2aec19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:42408/0 (socket says 172.21.15.67:42408) 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 -- 172.21.15.67:0/39577567 learned_addr learned my addr 172.21.15.67:0/39577567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2aea800640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108f90 0x7f2aec1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 -- 172.21.15.67:0/39577567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108f90 msgr2=0x7f2aec1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108f90 0x7f2aec1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 -- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec108610 msgr2=0x7f2aec19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec108610 0x7f2aec19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.703+0000 7f2ae9e00640 1 -- 172.21.15.67:0/39577567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec1a1210 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2ae9e00640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc032d50 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2adc0044d0 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec1a14a0 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc030c00 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.704+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1a7bc0 con 0x7f2aec102c00 2024-09-10T22:56:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.705+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec102fe0 con 0x7f2aec102c00 2024-09-10T22:56:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.708+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2adc004040 con 0x7f2aec102c00 2024-09-10T22:56:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.709+0000 7f2ae3400640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 0x7f2ad00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.709+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f2adc106af0 con 0x7f2aec102c00 2024-09-10T22:56:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.709+0000 7f2ae1400640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 0x7f2ad00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.710+0000 7f2ae1400640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 0x7f2ad00b7920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2aec101c10 tx=0x7f2ad4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.711+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2adc0cdd70 con 0x7f2aec102c00 2024-09-10T22:56:14.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:14.916+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2aec19c290 con 0x7f2aec102c00 2024-09-10T22:56:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:15.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.870+0000 7f2ae3400640 1 -- 172.21.15.67:0/39577567 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f2aec19c290 con 0x7f2aec102c00 2024-09-10T22:56:15.873 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-09-10T22:56:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 msgr2=0x7f2ad00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 0x7f2ad00b7920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2aec101c10 tx=0x7f2ad4005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 msgr2=0x7f2aec19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc003f30 comp rx=0 tx=0).stop 2024-09-10T22:56:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 shutdown_connections 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ad00b5460 0x7f2ad00b7920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aec108f90 0x7f2aec1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec108610 0x7f2aec19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 --2- 172.21.15.67:0/39577567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aec102c00 0x7f2aec19b160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 >> 172.21.15.67:0/39577567 conn(0x7f2aec0fe6d0 msgr2=0x7f2aec104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 shutdown_connections 2024-09-10T22:56:15.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:15.876+0000 7f2aeb200640 1 -- 172.21.15.67:0/39577567 wait complete. 2024-09-10T22:56:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/39577567' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:16.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/39577567' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:16.269 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/39577567' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:16.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:56:16.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T22:56:16.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:16 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:56:16.879 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:16 smithi067 ceph-mon[30939]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:56:17.079 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:17.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:16 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:56:17.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:16 smithi145 ceph-mon[34397]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:56:17.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:16 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:56:17.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:16 smithi092 ceph-mon[34223]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 -- 172.21.15.67:0/565836147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a81087d0 msgr2=0x7f81a8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 --2- 172.21.15.67:0/565836147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a81087d0 0x7f81a8108c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8194009900 tx=0x7f819402a410 comp rx=0 tx=0).stop 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 -- 172.21.15.67:0/565836147 shutdown_connections 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 --2- 172.21.15.67:0/565836147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8109150 0x7f81a810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 --2- 172.21.15.67:0/565836147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a81087d0 0x7f81a8108c10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 --2- 172.21.15.67:0/565836147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a8102dc0 0x7f81a81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 -- 172.21.15.67:0/565836147 >> 172.21.15.67:0/565836147 conn(0x7f81a80fe830 msgr2=0x7f81a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 -- 172.21.15.67:0/565836147 shutdown_connections 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.644+0000 7f81b0400640 1 -- 172.21.15.67:0/565836147 wait complete. 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.645+0000 7f81b0400640 1 Processor -- start 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.645+0000 7f81b0400640 1 -- start start 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.645+0000 7f81b0400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8102dc0 0x7f81a8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.645+0000 7f81b0400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a81087d0 0x7f81a81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81b0400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81b0400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112cd0 con 0x7f81a8109150 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81b0400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112b50 con 0x7f81a81087d0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81b0400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8112e50 con 0x7f81a8102dc0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38598/0 (socket says 172.21.15.67:38598) 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 -- 172.21.15.67:0/3023497774 learned_addr learned my addr 172.21.15.67:0/3023497774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ad000640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a81087d0 0x7f81a81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ada00640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8102dc0 0x7f81a8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 -- 172.21.15.67:0/3023497774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8102dc0 msgr2=0x7f81a8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8102dc0 0x7f81a8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 -- 172.21.15.67:0/3023497774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a81087d0 msgr2=0x7f81a81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a81087d0 0x7f81a81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.646+0000 7f81ae400640 1 -- 172.21.15.67:0/3023497774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a819f230 con 0x7f81a8109150 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81ae400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81a000eaa0 tx=0x7f81a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a000cdb0 con 0x7f81a8109150 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a819f520 con 0x7f81a8109150 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81a0005020 con 0x7f81a8109150 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00116d0 con 0x7f81a8109150 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.647+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a81a5db0 con 0x7f81a8109150 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.648+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a81031a0 con 0x7f81a8109150 2024-09-10T22:56:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.651+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f81a000f040 con 0x7f81a8109150 2024-09-10T22:56:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.652+0000 7f81a6000640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 0x7f81900b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.652+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7712+0+0 (secure 0 0 0) 0x7f81a00e5b50 con 0x7f81a8109150 2024-09-10T22:56:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.652+0000 7f81ada00640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 0x7f81900b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.653+0000 7f81ada00640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 0x7f81900b7870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f819c00bbe0 tx=0x7f819c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.654+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f81a0016030 con 0x7f81a8109150 2024-09-10T22:56:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.848+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f81a819a2b0 con 0x7f81a8109150 2024-09-10T22:56:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.884+0000 7f81a6000640 1 -- 172.21.15.67:0/3023497774 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7f81a00ac620 con 0x7f81a8109150 2024-09-10T22:56:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-09-10T22:56:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.886+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 msgr2=0x7f81900b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.886+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 0x7f81900b7870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f819c00bbe0 tx=0x7f819c00b390 comp rx=0 tx=0).stop 2024-09-10T22:56:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.886+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 msgr2=0x7f81a819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.886+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81a000eaa0 tx=0x7f81a000ef60 comp rx=0 tx=0).stop 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 shutdown_connections 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f81900b53b0 0x7f81900b7870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a8109150 0x7f81a819ea60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81a81087d0 0x7f81a81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 --2- 172.21.15.67:0/3023497774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a8102dc0 0x7f81a8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 >> 172.21.15.67:0/3023497774 conn(0x7f81a80fe830 msgr2=0x7f81a8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 shutdown_connections 2024-09-10T22:56:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:17.887+0000 7f81b0400640 1 -- 172.21.15.67:0/3023497774 wait complete. 2024-09-10T22:56:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:17 smithi145 ceph-mon[34397]: pgmap v295: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:17 smithi145 ceph-mon[34397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:18.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:17 smithi145 ceph-mon[34397]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:56:18.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:56:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:17 smithi092 ceph-mon[34223]: pgmap v295: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:17 smithi092 ceph-mon[34223]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:17 smithi092 ceph-mon[34223]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:56:18.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:56:18.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:17 smithi067 ceph-mon[30939]: pgmap v295: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:18.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:17 smithi067 ceph-mon[30939]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:18.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:17 smithi067 ceph-mon[30939]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:56:18.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:56:18.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:56:19.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:56:19.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:18 smithi067 ceph-mon[30939]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:56:19.091 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:56:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:18 smithi092 ceph-mon[34223]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:56:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3023497774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:56:19.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:18 smithi145 ceph-mon[34397]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.638+0000 7f5cd0a00640 1 -- 172.21.15.67:0/604409261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 msgr2=0x7f5ccc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.638+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/604409261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc1031a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc02a400 comp rx=0 tx=0).stop 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.638+0000 7f5cd0a00640 1 -- 172.21.15.67:0/604409261 shutdown_connections 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/604409261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 0x7f5ccc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/604409261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 0x7f5ccc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/604409261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc1031a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 -- 172.21.15.67:0/604409261 >> 172.21.15.67:0/604409261 conn(0x7f5ccc0fe830 msgr2=0x7f5ccc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 -- 172.21.15.67:0/604409261 shutdown_connections 2024-09-10T22:56:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 -- 172.21.15.67:0/604409261 wait complete. 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.639+0000 7f5cd0a00640 1 Processor -- start 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 -- start start 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 0x7f5ccc19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 0x7f5ccc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc112cc0 con 0x7f5ccc102dc0 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc112b40 con 0x7f5ccc109150 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cd0a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc112e40 con 0x7f5ccc1087d0 2024-09-10T22:56:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cc9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cc9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39022/0 (socket says 172.21.15.67:39022) 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cc9e00640 1 -- 172.21.15.67:0/4046624318 learned_addr learned my addr 172.21.15.67:0/4046624318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cc1400640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 0x7f5ccc19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.640+0000 7f5cca800640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 0x7f5ccc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 -- 172.21.15.67:0/4046624318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 msgr2=0x7f5ccc19b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 0x7f5ccc19b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 -- 172.21.15.67:0/4046624318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 msgr2=0x7f5ccc1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 0x7f5ccc1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 -- 172.21.15.67:0/4046624318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc1a13b0 con 0x7f5ccc102dc0 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc9e00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009a20 tx=0x7f5cbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc03f030 con 0x7f5ccc102dc0 2024-09-10T22:56:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc0035f0 con 0x7f5ccc102dc0 2024-09-10T22:56:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.641+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1a1640 con 0x7f5ccc102dc0 2024-09-10T22:56:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.642+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc035070 con 0x7f5ccc102dc0 2024-09-10T22:56:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.642+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a7d60 con 0x7f5ccc102dc0 2024-09-10T22:56:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.643+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc1031a0 con 0x7f5ccc102dc0 2024-09-10T22:56:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.645+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5cbc031a70 con 0x7f5ccc102dc0 2024-09-10T22:56:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.647+0000 7f5cc3400640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 0x7f5cb00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.647+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8094+0+0 (secure 0 0 0) 0x7f5cbc106a80 con 0x7f5ccc102dc0 2024-09-10T22:56:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.647+0000 7f5cc1400640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 0x7f5cb00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.648+0000 7f5cc1400640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 0x7f5cb00b7870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc101d20 tx=0x7f5cb400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.649+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5cbc0cd980 con 0x7f5ccc102dc0 2024-09-10T22:56:19.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.884+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5ccc1a7ff0 con 0x7f5ccc102dc0 2024-09-10T22:56:19.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.894+0000 7f5cc3400640 1 -- 172.21.15.67:0/4046624318 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5cbc0cd280 con 0x7f5ccc102dc0 2024-09-10T22:56:19.898 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.896+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 msgr2=0x7f5cb00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 0x7f5cb00b7870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc101d20 tx=0x7f5cb400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 msgr2=0x7f5ccc19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009a20 tx=0x7f5cbc004270 comp rx=0 tx=0).stop 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 shutdown_connections 2024-09-10T22:56:19.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cb00b53b0 0x7f5cb00b7870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ccc109150 0x7f5ccc1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ccc1087d0 0x7f5ccc19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 --2- 172.21.15.67:0/4046624318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ccc102dc0 0x7f5ccc19b300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.897+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 >> 172.21.15.67:0/4046624318 conn(0x7f5ccc0fe830 msgr2=0x7f5ccc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.898+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 shutdown_connections 2024-09-10T22:56:19.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:19.898+0000 7f5cd0a00640 1 -- 172.21.15.67:0/4046624318 wait complete. 2024-09-10T22:56:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:19 smithi092 ceph-mon[34223]: pgmap v298: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:19 smithi092 ceph-mon[34223]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:56:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:56:20.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:19 smithi092 ceph-mon[34223]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:56:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:19 smithi067 ceph-mon[30939]: pgmap v298: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:19 smithi067 ceph-mon[30939]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:56:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:56:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:19 smithi067 ceph-mon[30939]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:56:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:19 smithi145 ceph-mon[34397]: pgmap v298: 97 pgs: 10 creating+peering, 86 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:19 smithi145 ceph-mon[34397]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:56:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:56:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:19 smithi145 ceph-mon[34397]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:56:20.726 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T22:56:20.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T22:56:21.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:56:21.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:20 smithi067 ceph-mon[30939]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:56:21.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:20 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:56:21.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:20 smithi067 ceph-mon[30939]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:56:21.110 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:56:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:20 smithi092 ceph-mon[34223]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:56:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:20 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:56:21.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:20 smithi092 ceph-mon[34223]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:56:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4046624318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:56:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:20 smithi145 ceph-mon[34397]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:56:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:20 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:56:21.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:20 smithi145 ceph-mon[34397]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:56:21.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 -- 172.21.15.67:0/1754255773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410a9b0 msgr2=0x7f0d9410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 --2- 172.21.15.67:0/1754255773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410a9b0 0x7f0d9410adf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009900 tx=0x7f0d7c02a410 comp rx=0 tx=0).stop 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 -- 172.21.15.67:0/1754255773 shutdown_connections 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 --2- 172.21.15.67:0/1754255773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d9410b330 0x7f0d94111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 --2- 172.21.15.67:0/1754255773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410a9b0 0x7f0d9410adf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 --2- 172.21.15.67:0/1754255773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d94104fa0 0x7f0d94105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 -- 172.21.15.67:0/1754255773 >> 172.21.15.67:0/1754255773 conn(0x7f0d94100a50 msgr2=0x7f0d94102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 -- 172.21.15.67:0/1754255773 shutdown_connections 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.735+0000 7f0d99000640 1 -- 172.21.15.67:0/1754255773 wait complete. 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 Processor -- start 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 -- start start 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 0x7f0d9419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9410a9b0 0x7f0d9419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94114e10 con 0x7f0d9410b330 2024-09-10T22:56:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94114c90 con 0x7f0d9410a9b0 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.736+0000 7f0d99000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94114f90 con 0x7f0d94104fa0 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39040/0 (socket says 172.21.15.67:39040) 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 -- 172.21.15.67:0/126597225 learned_addr learned my addr 172.21.15.67:0/126597225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d89400640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9410a9b0 0x7f0d9419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d91e00640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 0x7f0d9419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 -- 172.21.15.67:0/126597225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 msgr2=0x7f0d9419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 0x7f0d9419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 -- 172.21.15.67:0/126597225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9410a9b0 msgr2=0x7f0d9419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9410a9b0 0x7f0d9419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 -- 172.21.15.67:0/126597225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d941a14d0 con 0x7f0d9410b330 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d91e00640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 0x7f0d9419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.737+0000 7f0d92800640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d750 tx=0x7f0d8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.738+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0d9410b330 2024-09-10T22:56:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.738+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d941a17c0 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.738+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d8c0054b0 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.738+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c004b90 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.738+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d941a7e80 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.739+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d94105380 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.741+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f0d8c004d30 con 0x7f0d9410b330 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.743+0000 7f0d8b400640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 0x7f0d780b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.743+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f0d8c0e55f0 con 0x7f0d9410b330 2024-09-10T22:56:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.743+0000 7f0d91e00640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 0x7f0d780b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.743+0000 7f0d91e00640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 0x7f0d780b7870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84009970 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.746+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0d8c0ac470 con 0x7f0d9410b330 2024-09-10T22:56:21.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:21.954+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0d9402e5c0 con 0x7f0d9410b330 2024-09-10T22:56:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: pgmap v301: 97 pgs: 1 creating+activating, 33 creating+peering, 9 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-10T22:56:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:56:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T22:56:22.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:active 2024-09-10T22:56:22.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:21 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: pgmap v301: 97 pgs: 1 creating+activating, 33 creating+peering, 9 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-10T22:56:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:56:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T22:56:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:active 2024-09-10T22:56:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:21 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: pgmap v301: 97 pgs: 1 creating+activating, 33 creating+peering, 9 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-10T22:56:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:56:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T22:56:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] up:active 2024-09-10T22:56:22.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:21 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:22.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.922+0000 7f0d8b400640 1 -- 172.21.15.67:0/126597225 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f0d8c0abd70 con 0x7f0d9410b330 2024-09-10T22:56:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 msgr2=0x7f0d780b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 0x7f0d780b7870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84009970 comp rx=0 tx=0).stop 2024-09-10T22:56:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 msgr2=0x7f0d941a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d750 tx=0x7f0d8c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:56:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 shutdown_connections 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0d780b53b0 0x7f0d780b7870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d9410b330 0x7f0d941a0d00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9410a9b0 0x7f0d9419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 --2- 172.21.15.67:0/126597225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d94104fa0 0x7f0d9419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 >> 172.21.15.67:0/126597225 conn(0x7f0d94100a50 msgr2=0x7f0d94102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.926+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 shutdown_connections 2024-09-10T22:56:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:22.927+0000 7f0d99000640 1 -- 172.21.15.67:0/126597225 wait complete. 2024-09-10T22:56:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:22 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:56:23.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:22 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:56:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:22 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:56:23.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:56:23.771 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:56:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: 2024-09-10T22:56:23.373+0000 7f544aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:56:24.099 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:24.238 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:23 smithi067 ceph-mon[30939]: pgmap v302: 97 pgs: 1 creating+activating, 30 creating+peering, 66 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s wr, 6 op/s 2024-09-10T22:56:24.238 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:23 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:56:24.238 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:23 smithi067 ceph-mon[30939]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:23 smithi092 ceph-mon[34223]: pgmap v302: 97 pgs: 1 creating+activating, 30 creating+peering, 66 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s wr, 6 op/s 2024-09-10T22:56:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:23 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:56:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:23 smithi092 ceph-mon[34223]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:23 smithi145 ceph-mon[34397]: pgmap v302: 97 pgs: 1 creating+activating, 30 creating+peering, 66 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s wr, 6 op/s 2024-09-10T22:56:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:23 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/126597225' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:56:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:23 smithi145 ceph-mon[34397]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:24.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 -- 172.21.15.67:0/3763782243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 msgr2=0x7fae240758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 --2- 172.21.15.67:0/3763782243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae240758a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae14009900 tx=0x7fae1402a410 comp rx=0 tx=0).stop 2024-09-10T22:56:24.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 -- 172.21.15.67:0/3763782243 shutdown_connections 2024-09-10T22:56:24.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 --2- 172.21.15.67:0/3763782243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 0x7fae24111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 --2- 172.21.15.67:0/3763782243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 --2- 172.21.15.67:0/3763782243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae240758a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 -- 172.21.15.67:0/3763782243 >> 172.21.15.67:0/3763782243 conn(0x7fae240fe660 msgr2=0x7fae24100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 -- 172.21.15.67:0/3763782243 shutdown_connections 2024-09-10T22:56:24.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.830+0000 7fae23e00640 1 -- 172.21.15.67:0/3763782243 wait complete. 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.831+0000 7fae23e00640 1 Processor -- start 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.831+0000 7fae23e00640 1 -- start start 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 0x7fae241a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae24114d60 con 0x7fae240754c0 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fae24114be0 con 0x7fae24075de0 2024-09-10T22:56:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae24114ee0 con 0x7fae2410f220 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39046/0 (socket says 172.21.15.67:39046) 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae22a00640 1 -- 172.21.15.67:0/2324224347 learned_addr learned my addr 172.21.15.67:0/2324224347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae23400640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 0x7fae241a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae1a000640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae22a00640 1 -- 172.21.15.67:0/2324224347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 msgr2=0x7fae241a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.832+0000 7fae22a00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 0x7fae241a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae22a00640 1 -- 172.21.15.67:0/2324224347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 msgr2=0x7fae2419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae22a00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae22a00640 1 -- 172.21.15.67:0/2324224347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae241a1240 con 0x7fae240754c0 2024-09-10T22:56:24.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae1a000640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae22a00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae1402a930 tx=0x7fae14003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1403f070 con 0x7fae240754c0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae1403bd00 con 0x7fae240754c0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14035030 con 0x7fae240754c0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae241a14d0 con 0x7fae240754c0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.833+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae241a7bf0 con 0x7fae240754c0 2024-09-10T22:56:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.834+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae240758a0 con 0x7fae240754c0 2024-09-10T22:56:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.837+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fae140315b0 con 0x7fae240754c0 2024-09-10T22:56:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.838+0000 7fae1be00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 0x7fae080b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.838+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8492+0+0 (secure 0 0 0) 0x7fae14106e90 con 0x7fae240754c0 2024-09-10T22:56:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.838+0000 7fae1a000640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 0x7fae080b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.839+0000 7fae1a000640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 0x7fae080b7920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae24100ea0 tx=0x7fae0c00a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:24.840+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fae140cdbf0 con 0x7fae240754c0 2024-09-10T22:56:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:25.037+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fae2419c2c0 con 0x7fae240754c0 2024-09-10T22:56:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:25 smithi092 ceph-mon[34223]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T22:56:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:25 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:56:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:25 smithi067 ceph-mon[30939]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T22:56:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:25 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:56:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:25 smithi145 ceph-mon[34397]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T22:56:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:25 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:56:26.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.932+0000 7fae1be00640 1 -- 172.21.15.67:0/2324224347 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7fae140cd4f0 con 0x7fae240754c0 2024-09-10T22:56:26.933 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2024-09-10T22:56:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.936+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 msgr2=0x7fae080b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.936+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 0x7fae080b7920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae24100ea0 tx=0x7fae0c00a5c0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.936+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 msgr2=0x7fae2419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:26.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.936+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae1402a930 tx=0x7fae14003de0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 shutdown_connections 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fae080b5460 0x7fae080b7920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2410f220 0x7fae241a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae24075de0 0x7fae2419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 --2- 172.21.15.67:0/2324224347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae240754c0 0x7fae2419b190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 >> 172.21.15.67:0/2324224347 conn(0x7fae240fe660 msgr2=0x7fae241113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:26.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 shutdown_connections 2024-09-10T22:56:26.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:26.937+0000 7fae23e00640 1 -- 172.21.15.67:0/2324224347 wait complete. 2024-09-10T22:56:27.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:26 smithi092 ceph-mon[34223]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:56:27.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:26 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:56:27.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:26 smithi092 ceph-mon[34223]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:56:27.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:26 smithi067 ceph-mon[30939]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:56:27.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:26 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:56:27.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:26 smithi067 ceph-mon[30939]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:56:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:26 smithi145 ceph-mon[34397]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:56:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:26 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2324224347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:56:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:26 smithi145 ceph-mon[34397]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:56:27.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T22:56:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:27 smithi092 ceph-mon[34223]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:56:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:27 smithi092 ceph-mon[34223]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:56:28.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:27 smithi067 ceph-mon[30939]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:56:28.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:27 smithi067 ceph-mon[30939]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:56:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:27 smithi145 ceph-mon[34397]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:56:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:27 smithi145 ceph-mon[34397]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 -- 172.21.15.67:0/1873667682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c109150 msgr2=0x7f0b0c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 --2- 172.21.15.67:0/1873667682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c109150 0x7f0b0c10f950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009ca0 tx=0x7f0b0402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 -- 172.21.15.67:0/1873667682 shutdown_connections 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 --2- 172.21.15.67:0/1873667682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c109150 0x7f0b0c10f950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 --2- 172.21.15.67:0/1873667682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 --2- 172.21.15.67:0/1873667682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.711+0000 7f0b13a00640 1 -- 172.21.15.67:0/1873667682 >> 172.21.15.67:0/1873667682 conn(0x7f0b0c0fe830 msgr2=0x7f0b0c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.712+0000 7f0b13a00640 1 -- 172.21.15.67:0/1873667682 shutdown_connections 2024-09-10T22:56:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.712+0000 7f0b13a00640 1 -- 172.21.15.67:0/1873667682 wait complete. 2024-09-10T22:56:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.712+0000 7f0b13a00640 1 Processor -- start 2024-09-10T22:56:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 -- start start 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 0x7f0b0c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112cd0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112b50 con 0x7f0b0c102dc0 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b13a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112e50 con 0x7f0b0c109150 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39060/0 (socket says 172.21.15.67:39060) 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b03e00640 1 -- 172.21.15.67:0/595314884 learned_addr learned my addr 172.21.15.67:0/595314884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b11000640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.713+0000 7f0b11a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 0x7f0b0c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 -- 172.21.15.67:0/595314884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 msgr2=0x7f0b0c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 0x7f0b0c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 -- 172.21.15.67:0/595314884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 msgr2=0x7f0b0c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 -- 172.21.15.67:0/595314884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1a7620 con 0x7f0b0c1087d0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b11a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 0x7f0b0c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b11000640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b03e00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d770 tx=0x7f0af400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4017070 con 0x7f0b0c1087d0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.714+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0af40054b0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.715+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a7910 con 0x7f0b0c1087d0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.715+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004c60 con 0x7f0b0c1087d0 2024-09-10T22:56:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.715+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a7e00 con 0x7f0b0c1087d0 2024-09-10T22:56:28.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.716+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c1031a0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.718+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f0af4004e00 con 0x7f0b0c1087d0 2024-09-10T22:56:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.721+0000 7f0b0a000640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 0x7f0af00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.721+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8985+0+0 (secure 0 0 0) 0x7f0af40e5d20 con 0x7f0b0c1087d0 2024-09-10T22:56:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.721+0000 7f0b11000640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 0x7f0af00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.721+0000 7f0b11000640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 0x7f0af00b77f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fc0 tx=0x7f0afc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.723+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0af40ac8f0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.920+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0b0c0767a0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.937+0000 7f0b0a000640 1 -- 172.21.15.67:0/595314884 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f0af40ac1f0 con 0x7f0b0c1087d0 2024-09-10T22:56:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T22:56:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.939+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 msgr2=0x7f0af00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.939+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 0x7f0af00b77f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fc0 tx=0x7f0afc005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.939+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 msgr2=0x7f0b0c0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.939+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d770 tx=0x7f0af400dc30 comp rx=0 tx=0).stop 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 shutdown_connections 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0af00b5330 0x7f0af00b77f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b0c109150 0x7f0b0c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0c1087d0 0x7f0b0c0759e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 --2- 172.21.15.67:0/595314884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c102dc0 0x7f0b0c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 >> 172.21.15.67:0/595314884 conn(0x7f0b0c0fe830 msgr2=0x7f0b0c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 shutdown_connections 2024-09-10T22:56:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:28.940+0000 7f0b13a00640 1 -- 172.21.15.67:0/595314884 wait complete. 2024-09-10T22:56:29.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:28 smithi092 ceph-mon[34223]: pgmap v309: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-10T22:56:29.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:28 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:56:29.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:28 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:56:29.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:28 smithi092 ceph-mon[34223]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:56:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:28 smithi067 ceph-mon[30939]: pgmap v309: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-10T22:56:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:28 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:56:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:28 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:56:29.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:28 smithi067 ceph-mon[30939]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:56:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:28 smithi145 ceph-mon[34397]: pgmap v309: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-10T22:56:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:28 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:56:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:28 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/595314884' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:56:29.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:28 smithi145 ceph-mon[34397]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:56:29.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T22:56:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:30.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:29 smithi067 ceph-mon[30939]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:29 smithi092 ceph-mon[34223]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:29 smithi145 ceph-mon[34397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 -- 172.21.15.67:0/3079736769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105ef0 msgr2=0x7f9d4810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 --2- 172.21.15.67:0/3079736769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105ef0 0x7f9d4810f930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009900 tx=0x7f9d4002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 -- 172.21.15.67:0/3079736769 shutdown_connections 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 --2- 172.21.15.67:0/3079736769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105ef0 0x7f9d4810f930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 --2- 172.21.15.67:0/3079736769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48105570 0x7f9d481059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 --2- 172.21.15.67:0/3079736769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48101220 0x7f9d48101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 -- 172.21.15.67:0/3079736769 >> 172.21.15.67:0/3079736769 conn(0x7f9d48078100 msgr2=0x7f9d480ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 -- 172.21.15.67:0/3079736769 shutdown_connections 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.710+0000 7f9d4f000640 1 -- 172.21.15.67:0/3079736769 wait complete. 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 Processor -- start 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 -- start start 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105570 0x7f9d48199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48105ef0 0x7f9d4819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48112cf0 con 0x7f9d48101220 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48112b70 con 0x7f9d48105ef0 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.711+0000 7f9d4f000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48112e70 con 0x7f9d48105570 2024-09-10T22:56:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54444/0 (socket says 172.21.15.67:54444) 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 -- 172.21.15.67:0/527724804 learned_addr learned my addr 172.21.15.67:0/527724804 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d3f400640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105570 0x7f9d48199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d4d000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48105ef0 0x7f9d4819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 -- 172.21.15.67:0/527724804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105570 msgr2=0x7f9d48199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105570 0x7f9d48199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 -- 172.21.15.67:0/527724804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48105ef0 msgr2=0x7f9d4819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48105ef0 0x7f9d4819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 -- 172.21.15.67:0/527724804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4819f270 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.712+0000 7f9d47e00640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b4a0 tx=0x7f9d3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.713+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38014070 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.713+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4819f560 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.713+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d38010aa0 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.713+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38017b70 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.713+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d481a5ca0 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.714+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48101600 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.716+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f9d38010610 con 0x7f9d48101220 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.719+0000 7f9d45600640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 0x7f9d2c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.719+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8985+0+0 (secure 0 0 0) 0x7f9d380e5940 con 0x7f9d48101220 2024-09-10T22:56:30.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.719+0000 7f9d3f400640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 0x7f9d2c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.719+0000 7f9d3f400640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 0x7f9d2c0b7870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d48108590 tx=0x7f9d30009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.721+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9d380ac4b0 con 0x7f9d48101220 2024-09-10T22:56:30.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.956+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f9d4819a2f0 con 0x7f9d48101220 2024-09-10T22:56:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.977+0000 7f9d45600640 1 -- 172.21.15.67:0/527724804 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f9d4819a2f0 con 0x7f9d48101220 2024-09-10T22:56:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2024-09-10T22:56:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 msgr2=0x7f9d2c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 0x7f9d2c0b7870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d48108590 tx=0x7f9d30009500 comp rx=0 tx=0).stop 2024-09-10T22:56:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 msgr2=0x7f9d481991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b4a0 tx=0x7f9d3800b960 comp rx=0 tx=0).stop 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 shutdown_connections 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d2c0b53b0 0x7f9d2c0b7870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d48105ef0 0x7f9d4819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d48105570 0x7f9d48199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 --2- 172.21.15.67:0/527724804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d48101220 0x7f9d481991c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 >> 172.21.15.67:0/527724804 conn(0x7f9d48078100 msgr2=0x7f9d480ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 shutdown_connections 2024-09-10T22:56:30.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:30.980+0000 7f9d4f000640 1 -- 172.21.15.67:0/527724804 wait complete. 2024-09-10T22:56:31.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:30 smithi092 ceph-mon[34223]: pgmap v311: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:56:31.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:30 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:56:31.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:30 smithi067 ceph-mon[30939]: pgmap v311: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:56:31.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:30 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:56:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:30 smithi145 ceph-mon[34397]: pgmap v311: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:56:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:30 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:56:31.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:32.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:31 smithi067 ceph-mon[30939]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:56:32.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:31 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:56:32.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:31 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:31 smithi092 ceph-mon[34223]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:56:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:31 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:56:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:31 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:31 smithi145 ceph-mon[34397]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:56:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:31 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/527724804' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:56:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:31 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:56:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 -- 172.21.15.67:0/2982388868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 msgr2=0x7fd3ac108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/2982388868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac108a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd39802a410 comp rx=0 tx=0).stop 2024-09-10T22:56:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 -- 172.21.15.67:0/2982388868 shutdown_connections 2024-09-10T22:56:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/2982388868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/2982388868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac108a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/2982388868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac102c10 0x7fd3ac102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 -- 172.21.15.67:0/2982388868 >> 172.21.15.67:0/2982388868 conn(0x7fd3ac0fe660 msgr2=0x7fd3ac100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 -- 172.21.15.67:0/2982388868 shutdown_connections 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.673+0000 7fd3b1e00640 1 -- 172.21.15.67:0/2982388868 wait complete. 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.674+0000 7fd3b1e00640 1 Processor -- start 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.674+0000 7fd3b1e00640 1 -- start start 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac102c10 0x7fd3ac198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac112b10 con 0x7fd3ac108620 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac112990 con 0x7fd3ac108fa0 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac112c90 con 0x7fd3ac102c10 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54462/0 (socket says 172.21.15.67:54462) 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 -- 172.21.15.67:0/3159330377 learned_addr learned my addr 172.21.15.67:0/3159330377 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b0a00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac102c10 0x7fd3ac198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1400640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 -- 172.21.15.67:0/3159330377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac102c10 msgr2=0x7fd3ac198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac102c10 0x7fd3ac198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 -- 172.21.15.67:0/3159330377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 msgr2=0x7fd3ac19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3abe00640 1 -- 172.21.15.67:0/3159330377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac19ef90 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.675+0000 7fd3b1400640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3abe00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd398009230 tx=0x7fd398004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39803f030 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd398003a80 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac19f220 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398035070 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.676+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a5940 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.677+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac102ff0 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.679+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fd39803bb80 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.681+0000 7fd3a9600640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 0x7fd3900b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.681+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fd398106d90 con 0x7fd3ac108620 2024-09-10T22:56:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.681+0000 7fd3b0a00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 0x7fd3900b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.682+0000 7fd3b0a00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 0x7fd3900b7920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00abe0 tx=0x7fd39c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.683+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd3980cd8e0 con 0x7fd3ac108620 2024-09-10T22:56:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.913+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3ac19a010 con 0x7fd3ac108620 2024-09-10T22:56:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.914+0000 7fd3a9600640 1 -- 172.21.15.67:0/3159330377 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11830 (secure 0 0 0) 0x7fd3ac19a010 con 0x7fd3ac108620 2024-09-10T22:56:32.917 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:32.917 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-09-10T22:56:30:969593+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:56:30.957536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34730,"qdb_cluster":[34730]},"id":1}]} 2024-09-10T22:56:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-09-10T22:56:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 msgr2=0x7fd3900b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 0x7fd3900b7920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00abe0 tx=0x7fd39c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 msgr2=0x7fd3ac199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd398009230 tx=0x7fd398004270 comp rx=0 tx=0).stop 2024-09-10T22:56:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 shutdown_connections 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3900b5460 0x7fd3900b7920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3ac108fa0 0x7fd3ac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac108620 0x7fd3ac199420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 --2- 172.21.15.67:0/3159330377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3ac102c10 0x7fd3ac198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 >> 172.21.15.67:0/3159330377 conn(0x7fd3ac0fe660 msgr2=0x7fd3ac101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 shutdown_connections 2024-09-10T22:56:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:32.917+0000 7fd3b1e00640 1 -- 172.21.15.67:0/3159330377 wait complete. 2024-09-10T22:56:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:32 smithi092 ceph-mon[34223]: pgmap v313: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:32 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3159330377' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:32 smithi067 ceph-mon[30939]: pgmap v313: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:33.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:32 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3159330377' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:32 smithi145 ceph-mon[34397]: pgmap v313: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:32 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3159330377' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:33.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:56:34.138 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:33 smithi092 ceph-mon[34223]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:56:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:33 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T22:56:34.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:33 smithi067 ceph-mon[30939]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:56:34.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:33 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T22:56:34.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:33 smithi145 ceph-mon[34397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:56:34.282 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:33 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T22:56:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 -- 172.21.15.67:0/600012173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 msgr2=0x7fcc1c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 --2- 172.21.15.67:0/600012173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c105380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009ca0 tx=0x7fcc0c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 -- 172.21.15.67:0/600012173 shutdown_connections 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 --2- 172.21.15.67:0/600012173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10b330 0x7fcc1c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 --2- 172.21.15.67:0/600012173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10a9b0 0x7fcc1c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 --2- 172.21.15.67:0/600012173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c105380 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 -- 172.21.15.67:0/600012173 >> 172.21.15.67:0/600012173 conn(0x7fcc1c100a50 msgr2=0x7fcc1c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 -- 172.21.15.67:0/600012173 shutdown_connections 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.835+0000 7fcc24200640 1 -- 172.21.15.67:0/600012173 wait complete. 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 Processor -- start 2024-09-10T22:56:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 -- start start 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10a9b0 0x7fcc1c19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10b330 0x7fcc1c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c114e40 con 0x7fcc1c104fa0 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c114cc0 con 0x7fcc1c10b330 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc24200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c114fc0 con 0x7fcc1c10a9b0 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc21800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.836+0000 7fcc21800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54482/0 (socket says 172.21.15.67:54482) 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 -- 172.21.15.67:0/2911472948 learned_addr learned my addr 172.21.15.67:0/2911472948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc22200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10b330 0x7fcc1c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc20e00640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10a9b0 0x7fcc1c19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 -- 172.21.15.67:0/2911472948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10a9b0 msgr2=0x7fcc1c19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10a9b0 0x7fcc1c19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 -- 172.21.15.67:0/2911472948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10b330 msgr2=0x7fcc1c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10b330 0x7fcc1c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 -- 172.21.15.67:0/2911472948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c1a1480 con 0x7fcc1c104fa0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.837+0000 7fcc21800640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c031040 tx=0x7fcc0c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.838+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c004040 con 0x7fcc1c104fa0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.838+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc0c004670 con 0x7fcc1c104fa0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.838+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c030d50 con 0x7fcc1c104fa0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.838+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1a1710 con 0x7fcc1c104fa0 2024-09-10T22:56:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.838+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1a7e30 con 0x7fcc1c104fa0 2024-09-10T22:56:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.839+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c105380 con 0x7fcc1c104fa0 2024-09-10T22:56:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.841+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fcc0c0041e0 con 0x7fcc1c104fa0 2024-09-10T22:56:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.842+0000 7fcc1a000640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 0x7fcc000b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.843+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fcc0c107230 con 0x7fcc1c104fa0 2024-09-10T22:56:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.843+0000 7fcc20e00640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 0x7fcc000b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.843+0000 7fcc20e00640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 0x7fcc000b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc08005fd0 tx=0x7fcc08005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:34.846+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcc0c0cdd80 con 0x7fcc1c104fa0 2024-09-10T22:56:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.043+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc1c19c500 con 0x7fcc1c104fa0 2024-09-10T22:56:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.045+0000 7fcc1a000640 1 -- 172.21.15.67:0/2911472948 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7fcc0c004820 con 0x7fcc1c104fa0 2024-09-10T22:56:35.047 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:35.047 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.047+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 msgr2=0x7fcc000b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.047+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 0x7fcc000b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc08005fd0 tx=0x7fcc08005850 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 msgr2=0x7fcc1c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c031040 tx=0x7fcc0c032bc0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 shutdown_connections 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcc000b53b0 0x7fcc000b7870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1c10b330 0x7fcc1c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc1c10a9b0 0x7fcc1c19b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 --2- 172.21.15.67:0/2911472948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc1c104fa0 0x7fcc1c19b3d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 >> 172.21.15.67:0/2911472948 conn(0x7fcc1c100a50 msgr2=0x7fcc1c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 shutdown_connections 2024-09-10T22:56:35.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:35.048+0000 7fcc24200640 1 -- 172.21.15.67:0/2911472948 wait complete. 2024-09-10T22:56:35.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:34 smithi092 ceph-mon[34223]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:35.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:34 smithi067 ceph-mon[30939]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:35.307 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:34 smithi145 ceph-mon[34397]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:35.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:56:36.227 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:35 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2911472948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:35 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2911472948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:35 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2911472948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:36.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 -- 172.21.15.67:0/4163920244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10a9d0 msgr2=0x7fe1bc10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/4163920244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc10ae10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009900 tx=0x7fe1a402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 -- 172.21.15.67:0/4163920244 shutdown_connections 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/4163920244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10b350 0x7fe1bc111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/4163920244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc10ae10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/4163920244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 -- 172.21.15.67:0/4163920244 >> 172.21.15.67:0/4163920244 conn(0x7fe1bc1009f0 msgr2=0x7fe1bc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 -- 172.21.15.67:0/4163920244 shutdown_connections 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.859+0000 7fe1c1c00640 1 -- 172.21.15.67:0/4163920244 wait complete. 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.860+0000 7fe1c1c00640 1 Processor -- start 2024-09-10T22:56:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 -- start start 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 0x7fe1bc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc114e80 con 0x7fe1bc10b350 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc114d00 con 0x7fe1bc104fc0 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1c1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc115000 con 0x7fe1bc10a9d0 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 0x7fe1bc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1ba000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1ba000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:45674/0 (socket says 172.21.15.67:45674) 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1ba000640 1 -- 172.21.15.67:0/2781475391 learned_addr learned my addr 172.21.15.67:0/2781475391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.861+0000 7fe1baa00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 -- 172.21.15.67:0/2781475391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 msgr2=0x7fe1bc19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 -- 172.21.15.67:0/2781475391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 msgr2=0x7fe1bc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 -- 172.21.15.67:0/2781475391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc1a14e0 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1baa00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1bb400640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 0x7fe1bc1a0d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b410 tx=0x7fe1b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4015020 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1b4011cc0 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1a17d0 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.862+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4010ce0 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.863+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a7e90 con 0x7fe1bc10b350 2024-09-10T22:56:36.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.864+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc1053a0 con 0x7fe1bc10b350 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.866+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe1b4011830 con 0x7fe1bc10b350 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.867+0000 7fe1b3400640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 0x7fe1a00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.867+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fe1b40e58b0 con 0x7fe1bc10b350 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.868+0000 7fe1baa00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 0x7fe1a00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.868+0000 7fe1baa00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 0x7fe1a00b7870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007e70 tx=0x7fe1ac007850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:36.871+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1b401f020 con 0x7fe1bc10b350 2024-09-10T22:56:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.070+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1bc02e5c0 con 0x7fe1bc10b350 2024-09-10T22:56:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.072+0000 7fe1b3400640 1 -- 172.21.15.67:0/2781475391 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7fe1b40ac400 con 0x7fe1bc10b350 2024-09-10T22:56:37.074 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:37.075 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 msgr2=0x7fe1a00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 0x7fe1a00b7870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007e70 tx=0x7fe1ac007850 comp rx=0 tx=0).stop 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 msgr2=0x7fe1bc1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 0x7fe1bc1a0d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b410 tx=0x7fe1b400b8d0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 shutdown_connections 2024-09-10T22:56:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1a00b53b0 0x7fe1a00b7870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1bc10b350 0x7fe1bc1a0d10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1bc10a9d0 0x7fe1bc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 --2- 172.21.15.67:0/2781475391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1bc104fc0 0x7fe1bc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 >> 172.21.15.67:0/2781475391 conn(0x7fe1bc1009f0 msgr2=0x7fe1bc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 shutdown_connections 2024-09-10T22:56:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:37.074+0000 7fe1c1c00640 1 -- 172.21.15.67:0/2781475391 wait complete. 2024-09-10T22:56:37.114 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:37 smithi067 ceph-mon[30939]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:37.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:37 smithi092 ceph-mon[34223]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:37.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:37 smithi145 ceph-mon[34397]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:38.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:38 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2781475391' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:38 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2781475391' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:38.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:38 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2781475391' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:38.459 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:39.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 -- 172.21.15.67:0/3709573170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10b320 msgr2=0x7f5eec111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:39.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/3709573170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10b320 0x7f5eec111b20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009ca0 tx=0x7f5ee402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 -- 172.21.15.67:0/3709573170 shutdown_connections 2024-09-10T22:56:39.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/3709573170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10b320 0x7f5eec111b20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/3709573170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec10a9a0 0x7f5eec10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/3709573170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec104f90 0x7f5eec105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 -- 172.21.15.67:0/3709573170 >> 172.21.15.67:0/3709573170 conn(0x7f5eec100a00 msgr2=0x7f5eec102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 -- 172.21.15.67:0/3709573170 shutdown_connections 2024-09-10T22:56:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.045+0000 7f5ef0e00640 1 -- 172.21.15.67:0/3709573170 wait complete. 2024-09-10T22:56:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 Processor -- start 2024-09-10T22:56:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 -- start start 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 0x7f5eec19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec10b320 0x7f5eec1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec114e50 con 0x7f5eec10a9a0 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec114cd0 con 0x7f5eec104f90 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.046+0000 7f5ef0e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec114fd0 con 0x7f5eec10b320 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 0x7f5eec19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49088/0 (socket says 172.21.15.67:49088) 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 0x7f5eec19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:57822/0 (socket says 172.21.15.67:57822) 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 -- 172.21.15.67:0/2173773630 learned_addr learned my addr 172.21.15.67:0/2173773630 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5eea800640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec10b320 0x7f5eec1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 -- 172.21.15.67:0/2173773630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec10b320 msgr2=0x7f5eec1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec10b320 0x7f5eec1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 -- 172.21.15.67:0/2173773630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 msgr2=0x7f5eec19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 0x7f5eec19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 -- 172.21.15.67:0/2173773630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec1a13f0 con 0x7f5eec10a9a0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.047+0000 7f5ee1400640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400d950 tx=0x7f5ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.048+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4017030 con 0x7f5eec10a9a0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.048+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec1a16e0 con 0x7f5eec10a9a0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.048+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ed4004020 con 0x7f5eec10a9a0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.048+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40055d0 con 0x7f5eec10a9a0 2024-09-10T22:56:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.048+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec1a7f70 con 0x7f5eec10a9a0 2024-09-10T22:56:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.049+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec105370 con 0x7f5eec10a9a0 2024-09-10T22:56:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.051+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5ed4009a30 con 0x7f5eec10a9a0 2024-09-10T22:56:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.052+0000 7f5ee3400640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 0x7f5ed00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.053+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f5ed40e5880 con 0x7f5eec10a9a0 2024-09-10T22:56:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.053+0000 7f5ee9e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 0x7f5ed00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.053+0000 7f5ee9e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 0x7f5ed00b7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005fd0 tx=0x7f5edc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.056+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ed40ac3d0 con 0x7f5eec10a9a0 2024-09-10T22:56:39.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.293+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eec10d130 con 0x7f5eec10a9a0 2024-09-10T22:56:39.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.294+0000 7f5ee3400640 1 -- 172.21.15.67:0/2173773630 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11830 (secure 0 0 0) 0x7f5ed40abcd0 con 0x7f5eec10a9a0 2024-09-10T22:56:39.296 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:39.296 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-09-10T22:56:30:969593+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24664,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24691,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34631,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34691,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:56:30.957536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34730,"qdb_cluster":[34730]},"id":1}]} 2024-09-10T22:56:39.296 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.296+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 msgr2=0x7f5ed00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.296+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 0x7f5ed00b7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005fd0 tx=0x7f5edc00b390 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.296+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 msgr2=0x7f5eec19b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.296+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400d950 tx=0x7f5ed400de10 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 shutdown_connections 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5ed00b53b0 0x7f5ed00b7870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eec10b320 0x7f5eec1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eec10a9a0 0x7f5eec19b880 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 --2- 172.21.15.67:0/2173773630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec104f90 0x7f5eec19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 >> 172.21.15.67:0/2173773630 conn(0x7f5eec100a00 msgr2=0x7f5eec101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 shutdown_connections 2024-09-10T22:56:39.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:39.297+0000 7f5ef0e00640 1 -- 172.21.15.67:0/2173773630 wait complete. 2024-09-10T22:56:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:39 smithi067 ceph-mon[30939]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:39.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:39 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2173773630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:39 smithi145 ceph-mon[34397]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:39 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2173773630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:39 smithi092 ceph-mon[34223]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:39 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2173773630' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:40.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:56:40.492 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:41.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 -- 172.21.15.67:0/3814055765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 msgr2=0x7fec4810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 --2- 172.21.15.67:0/3814055765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4810ade0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec3002a410 comp rx=0 tx=0).stop 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 -- 172.21.15.67:0/3814055765 shutdown_connections 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 --2- 172.21.15.67:0/3814055765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 0x7fec48111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 --2- 172.21.15.67:0/3814055765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4810ade0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 --2- 172.21.15.67:0/3814055765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec48105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 -- 172.21.15.67:0/3814055765 >> 172.21.15.67:0/3814055765 conn(0x7fec48100a00 msgr2=0x7fec48102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 -- 172.21.15.67:0/3814055765 shutdown_connections 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.127+0000 7fec4d000640 1 -- 172.21.15.67:0/3814055765 wait complete. 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 Processor -- start 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 -- start start 2024-09-10T22:56:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec4819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 0x7fec481a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fec48114e00 con 0x7fec4810a9a0 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec48114c80 con 0x7fec48104f90 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.128+0000 7fec4d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fec48114f80 con 0x7fec4810b320 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec45e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec4819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec45e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec4819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:57848/0 (socket says 172.21.15.67:57848) 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec45e00640 1 -- 172.21.15.67:0/1345604060 learned_addr learned my addr 172.21.15.67:0/1345604060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec46800640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 0x7fec481a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 -- 172.21.15.67:0/1345604060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 msgr2=0x7fec481a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 0x7fec481a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 -- 172.21.15.67:0/1345604060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 msgr2=0x7fec4819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec4819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.129+0000 7fec3d400640 1 -- 172.21.15.67:0/1345604060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec481a1420 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec3d400640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4819b8b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec30005940 tx=0x7fec300040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3003f070 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec30003c80 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30035070 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec481a16b0 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.130+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481a7dd0 con 0x7fec4810a9a0 2024-09-10T22:56:41.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.131+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec48105370 con 0x7fec4810a9a0 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.134+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fec3004b060 con 0x7fec4810a9a0 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.135+0000 7fec3f400640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 0x7fec2c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.135+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fec30106990 con 0x7fec4810a9a0 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.135+0000 7fec45e00640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 0x7fec2c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.136+0000 7fec45e00640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 0x7fec2c0b7870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec380104e0 tx=0x7fec38009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:41.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.137+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fec300cd4e0 con 0x7fec4810a9a0 2024-09-10T22:56:41.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.335+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec4802e5c0 con 0x7fec4810a9a0 2024-09-10T22:56:41.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.337+0000 7fec3f400640 1 -- 172.21.15.67:0/1345604060 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7fec300ccde0 con 0x7fec4810a9a0 2024-09-10T22:56:41.339 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:41.339 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:56:41.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 msgr2=0x7fec2c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:41.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 0x7fec2c0b7870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec380104e0 tx=0x7fec38009800 comp rx=0 tx=0).stop 2024-09-10T22:56:41.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 msgr2=0x7fec4819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:41.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4819b8b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec30005940 tx=0x7fec300040e0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 shutdown_connections 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec2c0b53b0 0x7fec2c0b7870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec4810b320 0x7fec481a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec4810a9a0 0x7fec4819b8b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 --2- 172.21.15.67:0/1345604060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec48104f90 0x7fec4819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.339+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 >> 172.21.15.67:0/1345604060 conn(0x7fec48100a00 msgr2=0x7fec48102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.340+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 shutdown_connections 2024-09-10T22:56:41.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:41.340+0000 7fec4d000640 1 -- 172.21.15.67:0/1345604060 wait complete. 2024-09-10T22:56:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:41 smithi145 ceph-mon[34397]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:41.694 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:41 smithi067 ceph-mon[30939]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:41 smithi092 ceph-mon[34223]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:42.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:56:19.886591+0000', 'modified': '2024-09-10T22:56:30.957536+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34730': {'gid': 34730, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/1949986898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1949986898}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1949986898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34730, 'qdb_cluster': [34730]} 2024-09-10T22:56:42.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:56:42.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:56:42.476 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:42.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:42 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1345604060' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:42.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:42 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1345604060' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:42.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:42 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1345604060' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:56:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 -- 172.21.15.67:0/2858960506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 msgr2=0x7f8744075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 --2- 172.21.15.67:0/2858960506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f8744075920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f873402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 -- 172.21.15.67:0/2858960506 shutdown_connections 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 --2- 172.21.15.67:0/2858960506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 --2- 172.21.15.67:0/2858960506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 0x7f874410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.047+0000 7f8749800640 1 --2- 172.21.15.67:0/2858960506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f8744075920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.048+0000 7f8749800640 1 -- 172.21.15.67:0/2858960506 >> 172.21.15.67:0/2858960506 conn(0x7f87440fe810 msgr2=0x7f8744100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.048+0000 7f8749800640 1 -- 172.21.15.67:0/2858960506 shutdown_connections 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.048+0000 7f8749800640 1 -- 172.21.15.67:0/2858960506 wait complete. 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.048+0000 7f8749800640 1 Processor -- start 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 -- start start 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 0x7f874419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8744114e80 con 0x7f8744075540 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8744114d00 con 0x7f8744075f40 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8749800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8744115000 con 0x7f874410f3c0 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8742a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8742a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49142/0 (socket says 172.21.15.67:49142) 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8742a00640 1 -- 172.21.15.67:0/3700011255 learned_addr learned my addr 172.21.15.67:0/3700011255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f873a000640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 0x7f874419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.049+0000 7f8743400640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 -- 172.21.15.67:0/3700011255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 msgr2=0x7f87441a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 -- 172.21.15.67:0/3700011255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 msgr2=0x7f874419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 0x7f874419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 -- 172.21.15.67:0/3700011255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441a14a0 con 0x7f8744075540 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8743400640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8742a00640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f873402a910 tx=0x7f8734003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f873be00640 1 -- 172.21.15.67:0/3700011255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873403f070 con 0x7f8744075540 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f873be00640 1 -- 172.21.15.67:0/3700011255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8734003e40 con 0x7f8744075540 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87441a1730 con 0x7f8744075540 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f873be00640 1 -- 172.21.15.67:0/3700011255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734035030 con 0x7f8744075540 2024-09-10T22:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.050+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441a7e50 con 0x7f8744075540 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 Processor -- start 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 -- start start 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f873be00640 1 -- 172.21.15.67:0/3700011255 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8734031400 con 0x7f8744075540 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7b60 0x560e8a0f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e8a05ed50 con 0x560e8a0f3af0 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e8a04e180 con 0x560e8a0e7b60 2024-09-10T22:56:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.054+0000 7f874c476740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e8a051f90 con 0x560e8a0e7240 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49156/0 (socket says 172.21.15.67:49156) 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f8742a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7b60 0x560e8a0f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 -- 172.21.15.67:0/966484825 learned_addr learned my addr 172.21.15.67:0/966484825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f8743400640 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 -- 172.21.15.67:0/966484825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 msgr2=0x560e8a0e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 -- 172.21.15.67:0/966484825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7b60 msgr2=0x560e8a0f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7b60 0x560e8a0f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.055+0000 7f873a000640 1 -- 172.21.15.67:0/966484825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8a018a70 con 0x560e8a0f3af0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f873a000640 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f872c00bd10 tx=0x7f872c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a342087efb72265d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f8733e00640 1 -- 172.21.15.67:0/966484825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c015070 con 0x560e8a0f3af0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f8733e00640 1 -- 172.21.15.67:0/966484825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f872c01a8f0 con 0x560e8a0f3af0 2024-09-10T22:56:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f8733e00640 1 -- 172.21.15.67:0/966484825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c0118e0 con 0x560e8a0f3af0 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f873be00640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 0x7f87280b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f873be00640 1 -- 172.21.15.67:0/3700011255 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f8734106b40 con 0x7f8744075540 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.056+0000 7f8743400640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 0x7f87280b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 -- 172.21.15.67:0/966484825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 msgr2=0x560e8a0f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f872c00bd10 tx=0x7f872c00f730 comp rx=0 tx=0).stop 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 -- 172.21.15.67:0/966484825 shutdown_connections 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0f5ee0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7b60 0x560e8a0f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 --2- 172.21.15.67:0/966484825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 -- 172.21.15.67:0/966484825 >> 172.21.15.67:0/966484825 conn(0x560e8a0e5020 msgr2=0x560e8a0e6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f8743400640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 0x7f87280b77f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f874406c0d0 tx=0x7f873c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 -- 172.21.15.67:0/966484825 shutdown_connections 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.057+0000 7f874c476740 1 -- 172.21.15.67:0/966484825 wait complete. 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 -- start start 2024-09-10T22:56:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 Processor -- start 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a0f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e89eaf5f0 con 0x560e8a0e7b60 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e89fe9460 con 0x560e8a0f3af0 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e8a0f6840 con 0x560e8a0e7240 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.058+0000 7f874c476740 10 client.0 _ll_register_callbacks cb 0x560e89ff9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49164/0 (socket says 172.21.15.67:49164) 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 -- 172.21.15.67:0/74479215 learned_addr learned my addr 172.21.15.67:0/74479215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f8742a00640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f8743400640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a0f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 -- 172.21.15.67:0/74479215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 msgr2=0x560e8a0f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 -- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 msgr2=0x560e8a0f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a0f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 -- 172.21.15.67:0/74479215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e89dac540 con 0x560e8a0e7b60 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f873a000640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f872c000c00 tx=0x7f872c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a7645382bc7e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.059+0000 7f8730c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c027060 con 0x560e8a0e7b60 2024-09-10T22:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e89feba70 con 0x560e8a0e7b60 2024-09-10T22:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f872c004440 con 0x560e8a0e7b60 2024-09-10T22:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f872c02e050 con 0x560e8a0e7b60 2024-09-10T22:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f874c476740 10 client.14973 fetch_fsmap learned FSMap version 17 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x560e8a0066f0 con 0x560e8a0e7b60 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.060+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560e89feba70 con 0x560e8a0e7b60 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f872c004c20 con 0x560e8a0e7b60 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 10 client.14973 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 10 client.14973 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 20 client.14973 populate_metadata read hostname 'smithi067' 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a104430 0x560e8a1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 4 client.14973 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e8a060ca0 con 0x560e8a104430 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.061+0000 7f8742a00640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a104430 0x560e8a1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.062+0000 7f8742a00640 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a104430 0x560e8a1058a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e7118870c759ef server_cookie=e6cb15b5a2d3e5d6 in_seq=0 out_seq=0 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.062+0000 7f8730c00640 10 client.14973 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:56:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.063+0000 7f8730c00640 1 -- 172.21.15.67:0/74479215 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e8a060ca0 con 0x560e8a104430 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.063+0000 7f8730c00640 10 client.14973 handle_command_reply: tid=0 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.063+0000 7f874c476740 1 client.14973 shutdown 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 msgr2=0x560e8a0f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f872c000c00 tx=0x7f872c011440 comp rx=0 tx=0).stop 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 shutdown_connections 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a104430 0x560e8a1058a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a0f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7b60 0x560e8a0f0f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 --2- 172.21.15.67:0/74479215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7240 0x560e8a0f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 >> 172.21.15.67:0/74479215 conn(0x560e8a0e5020 msgr2=0x560e8a0eac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f8730c00640 0 client.14973 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 shutdown_connections 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 1 -- 172.21.15.67:0/74479215 wait complete. 2024-09-10T22:56:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.064+0000 7f874c476740 20 client.14973 trim_cache size 0 max 16384 2024-09-10T22:56:43.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.084+0000 7f874c476740 1 Processor -- start 2024-09-10T22:56:43.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.084+0000 7f874c476740 1 -- start start 2024-09-10T22:56:43.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7240 0x560e8a0eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e8a103be0 con 0x560e8a0f3af0 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e8a1042b0 con 0x560e8a0e7240 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f874c476740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e8a104980 con 0x560e8a0e7b60 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f8743400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f8743400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49168/0 (socket says 172.21.15.67:49168) 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f8743400640 1 -- 172.21.15.67:0/3631083325 learned_addr learned my addr 172.21.15.67:0/3631083325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f8742a00640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7240 0x560e8a0eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.085+0000 7f873a000640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 -- 172.21.15.67:0/3631083325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 msgr2=0x560e8a1058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a1058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 -- 172.21.15.67:0/3631083325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7240 msgr2=0x560e8a0eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7240 0x560e8a0eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 -- 172.21.15.67:0/3631083325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8a0f0240 con 0x560e8a0f3af0 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f873a000640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8743400640 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f873c012830 tx=0x7f873c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af84f727c704491 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8733400640 1 -- 172.21.15.67:0/3631083325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c004d20 con 0x560e8a0f3af0 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8733400640 1 -- 172.21.15.67:0/3631083325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f873c01fd50 con 0x560e8a0f3af0 2024-09-10T22:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.086+0000 7f8733400640 1 -- 172.21.15.67:0/3631083325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c005740 con 0x560e8a0f3af0 2024-09-10T22:56:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 -- 172.21.15.67:0/3631083325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 msgr2=0x560e8a0ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f873c012830 tx=0x7f873c014000 comp rx=0 tx=0).stop 2024-09-10T22:56:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 -- 172.21.15.67:0/3631083325 shutdown_connections 2024-09-10T22:56:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0f3af0 0x560e8a0ef2a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 --2- 172.21.15.67:0/3631083325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0e7240 0x560e8a0eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 -- 172.21.15.67:0/3631083325 >> 172.21.15.67:0/3631083325 conn(0x560e8a0e5020 msgr2=0x560e8a0ebc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 -- 172.21.15.67:0/3631083325 shutdown_connections 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.087+0000 7f874c476740 1 -- 172.21.15.67:0/3631083325 wait complete. 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 -- start start 2024-09-10T22:56:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 Processor -- start 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e8a105720 con 0x560e8a0e7240 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e8a105050 con 0x560e8a0f3af0 2024-09-10T22:56:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e8a105df0 con 0x560e8a0e7b60 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f874c476740 10 client.0 _ll_register_callbacks cb 0x7f872c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.088+0000 7f8742a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49180/0 (socket says 172.21.15.67:49180) 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 -- 172.21.15.67:0/268785580 learned_addr learned my addr 172.21.15.67:0/268785580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8743400640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f873a000640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 -- 172.21.15.67:0/268785580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 msgr2=0x560e8a108940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a108940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 -- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 msgr2=0x560e8a10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 -- 172.21.15.67:0/268785580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e8a10e0f0 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8742a00640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87340ccf20 tx=0x7f87341041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6762f00983a962d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8730c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734049040 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f87340d6090 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.089+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873c01fd50 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f87340cc8e0 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 10 client.14979 fetch_fsmap learned FSMap version 17 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x560e8a1717e0 con 0x560e8a0e7240 2024-09-10T22:56:43.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f873c01fd50 con 0x560e8a0e7240 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f87340b40b0 con 0x560e8a0e7240 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 10 client.14979 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 20 client.14979 populate_metadata read hostname 'smithi067' 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a176360 0x560e8a175070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 4 client.14979 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.090+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "status"}) -- 0x560e8a05c750 con 0x560e8a176360 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.091+0000 7f873a000640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a176360 0x560e8a175070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.091+0000 7f873a000640 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a176360 0x560e8a175070 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f17d0a0aca72046 server_cookie=2bcf8dd05f916002 in_seq=0 out_seq=0 2024-09-10T22:56:43.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.091+0000 7f8730c00640 10 client.14979 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f8730c00640 1 -- 172.21.15.67:0/268785580 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560e8a05c750 con 0x560e8a176360 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f8730c00640 10 client.14979 handle_command_reply: tid=0 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 client.14979 shutdown 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 msgr2=0x560e8a1083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87340ccf20 tx=0x7f87341041a0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 shutdown_connections 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e8a176360 0x560e8a175070 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a0f3af0 0x560e8a10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e8a0e7b60 0x560e8a108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 --2- 172.21.15.67:0/268785580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e8a0e7240 0x560e8a1083c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 >> 172.21.15.67:0/268785580 conn(0x560e8a0e5020 msgr2=0x560e8a0ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f8730c00640 0 client.14979 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 shutdown_connections 2024-09-10T22:56:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 1 -- 172.21.15.67:0/268785580 wait complete. 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.092+0000 7f874c476740 20 client.14979 trim_cache size 0 max 16384 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34730, 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:56:43.105 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 23.195012994999999, 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 35.547578420999997 2024-09-10T22:56:43.106 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 msgr2=0x7f87280b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 0x7f87280b77f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f874406c0d0 tx=0x7f873c009680 comp rx=0 tx=0).stop 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 msgr2=0x7f874419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f873402a910 tx=0x7f8734003c20 comp rx=0 tx=0).stop 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 shutdown_connections 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f87280b5330 0x7f87280b77f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874410f3c0 0x7f87441a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8744075f40 0x7f874419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 --2- 172.21.15.67:0/3700011255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8744075540 0x7f874419b3f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:43.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 >> 172.21.15.67:0/3700011255 conn(0x7f87440fe810 msgr2=0x7f87440ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:43.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 shutdown_connections 2024-09-10T22:56:43.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:43.093+0000 7f8749800640 1 -- 172.21.15.67:0/3700011255 wait complete. 2024-09-10T22:56:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:43 smithi145 ceph-mon[34397]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:43 smithi092 ceph-mon[34223]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:43.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:43 smithi067 ceph-mon[30939]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:43.931 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:56:43.931 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:43.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:56:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3719917917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010a9d0 msgr2=0x7f5fd010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3719917917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010a9d0 0x7f5fd010ae10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009900 tx=0x7f5fb802a410 comp rx=0 tx=0).stop 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3719917917 shutdown_connections 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3719917917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010b350 0x7f5fd0111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3719917917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010a9d0 0x7f5fd010ae10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3719917917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 0x7f5fd01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:44.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3719917917 >> 172.21.15.67:0/3719917917 conn(0x7f5fd01009f0 msgr2=0x7f5fd0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3719917917 shutdown_connections 2024-09-10T22:56:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.940+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3719917917 wait complete. 2024-09-10T22:56:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.941+0000 7f5fd6e00640 1 Processor -- start 2024-09-10T22:56:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 -- start start 2024-09-10T22:56:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 0x7f5fd019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010a9d0 0x7f5fd019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0114e50 con 0x7f5fd010b350 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0114cd0 con 0x7f5fd010a9d0 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd6e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0114fd0 con 0x7f5fd0104fc0 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49202/0 (socket says 172.21.15.67:49202) 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fd4e00640 1 -- 172.21.15.67:0/3511967619 learned_addr learned my addr 172.21.15.67:0/3511967619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.942+0000 7f5fcfe00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 0x7f5fd019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fcf400640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010a9d0 0x7f5fd019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 -- 172.21.15.67:0/3511967619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 msgr2=0x7f5fd019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 0x7f5fd019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 -- 172.21.15.67:0/3511967619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010a9d0 msgr2=0x7f5fd019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010a9d0 0x7f5fd019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 -- 172.21.15.67:0/3511967619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd01a1430 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fd4e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800d750 tx=0x7f5fc800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.943+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8017070 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.944+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc80054b0 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.944+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004b90 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.944+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd01a1720 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.944+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a7de0 con 0x7f5fd010b350 2024-09-10T22:56:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.945+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd01053a0 con 0x7f5fd010b350 2024-09-10T22:56:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.947+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5fc8004d30 con 0x7f5fd010b350 2024-09-10T22:56:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.949+0000 7f5fccc00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 0x7f5fb40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.949+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f5fc80e5c70 con 0x7f5fd010b350 2024-09-10T22:56:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.949+0000 7f5fcfe00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 0x7f5fb40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.950+0000 7f5fcfe00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 0x7f5fb40b7870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005fd0 tx=0x7f5fc0009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:44.951+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5fc80ac1f0 con 0x7f5fd010b350 2024-09-10T22:56:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.186+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5fd019c4b0 con 0x7f5fd010b350 2024-09-10T22:56:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.186+0000 7f5fccc00640 1 -- 172.21.15.67:0/3511967619 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f5fd019c4b0 con 0x7f5fd010b350 2024-09-10T22:56:45.188 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T22:56:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.188+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 msgr2=0x7f5fb40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.188+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 0x7f5fb40b7870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005fd0 tx=0x7f5fc0009760 comp rx=0 tx=0).stop 2024-09-10T22:56:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.188+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 msgr2=0x7f5fd01a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.188+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800d750 tx=0x7f5fc800dc10 comp rx=0 tx=0).stop 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 shutdown_connections 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5fb40b53b0 0x7f5fb40b7870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd010b350 0x7f5fd01a0c60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd010a9d0 0x7f5fd019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 --2- 172.21.15.67:0/3511967619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fd0104fc0 0x7f5fd019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 >> 172.21.15.67:0/3511967619 conn(0x7f5fd01009f0 msgr2=0x7f5fd0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 shutdown_connections 2024-09-10T22:56:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:45.189+0000 7f5fd6e00640 1 -- 172.21.15.67:0/3511967619 wait complete. 2024-09-10T22:56:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:45 smithi067 ceph-mon[30939]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:45 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3511967619' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:56:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:45 smithi145 ceph-mon[34397]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:45 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3511967619' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:56:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:45 smithi092 ceph-mon[34223]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:45 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3511967619' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:56:45.984 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:56:45.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T22:56:45.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:45.985 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:56:46.022 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42100sec preferred_lft 42100sec 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-10T22:56:46.023 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:56:46.024 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:56:46.024 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:56:46.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:56:46.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:46.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:46.442 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:56:46.499 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-09-10T22:56:46.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:46.503 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-09-10T22:56:46.550 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-09-10T22:56:46.550 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:56:46.550 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:56:46.550 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:56:46.551 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:56:46.552 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:56:46.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:56:47.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:47.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:47.216 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-10T22:56:47.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:47.252 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-09-10T22:56:47.329 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:56:47.330 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:56:47.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:47.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T22:56:47.330 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:56:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:56:47.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:47 smithi067 ceph-mon[30939]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:47.509 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T22:56:47.509 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T22:56:47.510 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:56:47.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:56:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:47 smithi145 ceph-mon[34397]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:47.756 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:56:47.757 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:56:47.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-09-10T22:56:47.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:56:47.757 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:56:47.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:47 smithi092 ceph-mon[34223]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:47.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:56:47.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:47.925 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T22:56:47.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:56:47.925 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:47.961 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T22:56:47.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:56:47.961 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:48.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-10T22:56:48.133 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-10T22:56:48.170 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-10T22:56:48.171 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/85b4c408ab335fb671d5a6c427ee879da0d5f3c2810ac54e08e3651d6f939a01/merged 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/485979c99d9cd1cd976a54e0849a94a21ab742c1535bfd3fefdaa4ce13d5b15e/merged 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:48.172 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/87080adde8a92b1932f774ea006d4c27dbc818ba6dfff8ad60857a334213e553/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3bb0ab0a2a9b9b7981bcc3c7f5f6a51952a3b06915c4c51c9d5e00c29ffa67/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/c306b63195b5a13ec2bde412cee798780c375c1aceb3af80f9ff4b3b76e8d017/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f52f248cff3d648d9c42c1aa1a916c75070ed8cf28516834061492f947436e4/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/411a4a8be1023dc81d20821921a9a2ca0236db2423f190a6e88942bd45a21363/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/ad9e8fd555e64eebea1a315c8ef7e47b2370ec6478a8e456aa0fd860a845750e/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/641cbf72fd4f1b70475507f3ec5d30a4b6fcfc8e187481db8e971de2e423d5be/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f7948be11e2d6942df6aab6a32015b6478914f0b821d683b9c859ced3abaf4ab/merged 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:48.173 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:48.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:48.174 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-10T22:56:48.250 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:56:48.250 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T22:56:48.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-10T22:56:48.352 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:56:48.389 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-10T22:56:48.389 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-10T22:56:48.389 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-10T22:56:48.389 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-10T22:56:48.390 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/85b4c408ab335fb671d5a6c427ee879da0d5f3c2810ac54e08e3651d6f939a01/merged 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/485979c99d9cd1cd976a54e0849a94a21ab742c1535bfd3fefdaa4ce13d5b15e/merged 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:48.391 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/87080adde8a92b1932f774ea006d4c27dbc818ba6dfff8ad60857a334213e553/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3bb0ab0a2a9b9b7981bcc3c7f5f6a51952a3b06915c4c51c9d5e00c29ffa67/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/c306b63195b5a13ec2bde412cee798780c375c1aceb3af80f9ff4b3b76e8d017/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f52f248cff3d648d9c42c1aa1a916c75070ed8cf28516834061492f947436e4/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/411a4a8be1023dc81d20821921a9a2ca0236db2423f190a6e88942bd45a21363/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/ad9e8fd555e64eebea1a315c8ef7e47b2370ec6478a8e456aa0fd860a845750e/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/641cbf72fd4f1b70475507f3ec5d30a4b6fcfc8e187481db8e971de2e423d5be/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f7948be11e2d6942df6aab6a32015b6478914f0b821d683b9c859ced3abaf4ab/merged 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:48.392 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:48.393 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-10T22:56:48.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2024-09-10T22:56:48.485+0000 7f26dfbd8480 -1 init, newargv = 0x55efc21ddfb0 newargc=15 2024-09-10T22:56:48.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[78848]: starting ceph client 2024-09-10T22:56:48.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[78848]: starting fuse 2024-09-10T22:56:49.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-10T22:56:49.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:49 smithi067 ceph-mon[30939]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:49 smithi145 ceph-mon[34397]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:49.635 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:56:49.673 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-10T22:56:49.673 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-10T22:56:49.673 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-10T22:56:49.673 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-10T22:56:49.673 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-10T22:56:49.674 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/85b4c408ab335fb671d5a6c427ee879da0d5f3c2810ac54e08e3651d6f939a01/merged 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/485979c99d9cd1cd976a54e0849a94a21ab742c1535bfd3fefdaa4ce13d5b15e/merged 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/87080adde8a92b1932f774ea006d4c27dbc818ba6dfff8ad60857a334213e553/merged 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3bb0ab0a2a9b9b7981bcc3c7f5f6a51952a3b06915c4c51c9d5e00c29ffa67/merged 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/c306b63195b5a13ec2bde412cee798780c375c1aceb3af80f9ff4b3b76e8d017/merged 2024-09-10T22:56:49.675 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f52f248cff3d648d9c42c1aa1a916c75070ed8cf28516834061492f947436e4/merged 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/411a4a8be1023dc81d20821921a9a2ca0236db2423f190a6e88942bd45a21363/merged 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/ad9e8fd555e64eebea1a315c8ef7e47b2370ec6478a8e456aa0fd860a845750e/merged 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/641cbf72fd4f1b70475507f3ec5d30a4b6fcfc8e187481db8e971de2e423d5be/merged 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f7948be11e2d6942df6aab6a32015b6478914f0b821d683b9c859ced3abaf4ab/merged 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:56:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:49.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:49.677 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-10T22:56:49.754 INFO:teuthology.orchestra.run.smithi067.stdout:120 2024-09-10T22:56:49.754 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:56:49.755 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2024-09-10T22:56:49.756 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-09-10T22:56:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:49 smithi092 ceph-mon[34223]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.78848.asok 2024-09-10T22:56:49.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:49.938 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78848.asok 2024-09-10T22:56:49.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:49.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78848.asok status 2024-09-10T22:56:50.120 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "78848", 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-09-10T22:56:50.121 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 14991, 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 14991 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 9273455 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:56:50.122 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 9273455 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.14991 192.168.0.1:0/9273455", 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/9273455", 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 85, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-09-10T22:56:50.123 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:56:50.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:50.133 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-09-10T22:56:50.133 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:50.170 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-09-10T22:56:50.170 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:50.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:50.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:50.269 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T22:56:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:56:50.378 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2024-09-10T22:56:50.378 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T22:56:50.379 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:56:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:56:50.387 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:56:50.387 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:56:50.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:56:50.841 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:51.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 -- 172.21.15.67:0/1723899620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 msgr2=0x7f4aac113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1723899620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac113440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009900 tx=0x7f4a9402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 -- 172.21.15.67:0/1723899620 shutdown_connections 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1723899620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac113980 0x7f4aac115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1723899620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac113440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1723899620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac0776d0 0x7f4aac077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 -- 172.21.15.67:0/1723899620 >> 172.21.15.67:0/1723899620 conn(0x7f4aac100a00 msgr2=0x7f4aac102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 -- 172.21.15.67:0/1723899620 shutdown_connections 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.431+0000 7f4ab1000640 1 -- 172.21.15.67:0/1723899620 wait complete. 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 Processor -- start 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 -- start start 2024-09-10T22:56:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac0776d0 0x7f4aac19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac113980 0x7f4aac1a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac119290 con 0x7f4aac077ff0 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac119110 con 0x7f4aac0776d0 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.432+0000 7f4ab1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac119410 con 0x7f4aac113980 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac0776d0 0x7f4aac19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57226/0 (socket says 172.21.15.67:57226) 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 -- 172.21.15.67:0/1004840919 learned_addr learned my addr 172.21.15.67:0/1004840919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aaa800640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac113980 0x7f4aac1a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 -- 172.21.15.67:0/1004840919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac113980 msgr2=0x7f4aac1a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac113980 0x7f4aac1a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 -- 172.21.15.67:0/1004840919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac0776d0 msgr2=0x7f4aac19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac0776d0 0x7f4aac19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.433+0000 7f4aa1400640 1 -- 172.21.15.67:0/1004840919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac1a5830 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4aa1400640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009a20 tx=0x7f4a94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9403f030 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac1a5ac0 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a940035f0 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94035070 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.434+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1ac1e0 con 0x7f4aac077ff0 2024-09-10T22:56:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.435+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac077ab0 con 0x7f4aac077ff0 2024-09-10T22:56:51.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.437+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f4a94031c80 con 0x7f4aac077ff0 2024-09-10T22:56:51.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.439+0000 7f4aa3400640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 0x7f4a900b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.439+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f4a94106db0 con 0x7f4aac077ff0 2024-09-10T22:56:51.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.439+0000 7f4aa9e00640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 0x7f4a900b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.440+0000 7f4aa9e00640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 0x7f4a900b7870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009990 tx=0x7f4a9c009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.441+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a940cd900 con 0x7f4aac077ff0 2024-09-10T22:56:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:51 smithi145 ceph-mon[34397]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.650+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4aac1a08b0 con 0x7f4aac077ff0 2024-09-10T22:56:51.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:51 smithi067 ceph-mon[30939]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.650+0000 7f4aa3400640 1 -- 172.21.15.67:0/1004840919 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+882 (secure 0 0 0) 0x7f4aac1a08b0 con 0x7f4aac077ff0 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2682299290 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3212905705 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/82707279 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3042343451 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3255023915 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2935874097 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:51.653 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3802899292 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3004982929 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1673435000 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-09-10T22:56:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 msgr2=0x7f4a900b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 0x7f4a900b7870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009990 tx=0x7f4a9c009110 comp rx=0 tx=0).stop 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 msgr2=0x7f4aac19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009a20 tx=0x7f4a94004270 comp rx=0 tx=0).stop 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 shutdown_connections 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4a900b53b0 0x7f4a900b7870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac113980 0x7f4aac1a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4aac077ff0 0x7f4aac19fcc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 --2- 172.21.15.67:0/1004840919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac0776d0 0x7f4aac19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 >> 172.21.15.67:0/1004840919 conn(0x7f4aac100a00 msgr2=0x7f4aac102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 shutdown_connections 2024-09-10T22:56:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:51.653+0000 7f4ab1000640 1 -- 172.21.15.67:0/1004840919 wait complete. 2024-09-10T22:56:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:51 smithi092 ceph-mon[34223]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:56:52.405 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:56:52.405 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:56:52.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:56:52.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:52 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1004840919' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:52.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:52 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1004840919' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:52.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:52 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1004840919' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:52.860 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 -- 172.21.15.67:0/1166772345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010a820 msgr2=0x7f3a0010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 --2- 172.21.15.67:0/1166772345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010a820 0x7f3a0010ac60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009900 tx=0x7f39ec02a400 comp rx=0 tx=0).stop 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 -- 172.21.15.67:0/1166772345 shutdown_connections 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 --2- 172.21.15.67:0/1166772345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a0010b1a0 0x7f3a001119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 --2- 172.21.15.67:0/1166772345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010a820 0x7f3a0010ac60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 --2- 172.21.15.67:0/1166772345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a00104e10 0x7f3a001051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 -- 172.21.15.67:0/1166772345 >> 172.21.15.67:0/1166772345 conn(0x7f3a00100840 msgr2=0x7f3a00102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 -- 172.21.15.67:0/1166772345 shutdown_connections 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.442+0000 7f3a06400640 1 -- 172.21.15.67:0/1166772345 wait complete. 2024-09-10T22:56:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.443+0000 7f3a06400640 1 Processor -- start 2024-09-10T22:56:53.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.443+0000 7f3a06400640 1 -- start start 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 0x7f3a0019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a0010a820 0x7f3a0019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00114d10 con 0x7f3a0010b1a0 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00114b90 con 0x7f3a00104e10 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a06400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00114e90 con 0x7f3a0010a820 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57250/0 (socket says 172.21.15.67:57250) 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 -- 172.21.15.67:0/725633946 learned_addr learned my addr 172.21.15.67:0/725633946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05000640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 0x7f3a0019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f39ffe00640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a0010a820 0x7f3a0019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 -- 172.21.15.67:0/725633946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a0010a820 msgr2=0x7f3a0019b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a0010a820 0x7f3a0019b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 -- 172.21.15.67:0/725633946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 msgr2=0x7f3a0019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 0x7f3a0019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05a00640 1 -- 172.21.15.67:0/725633946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a001a1150 con 0x7f3a0010b1a0 2024-09-10T22:56:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.444+0000 7f3a05000640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 0x7f3a0019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f3a05a00640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39f800eaa0 tx=0x7f39f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f800cdb0 con 0x7f3a0010b1a0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a001a1440 con 0x7f3a0010b1a0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39f8005020 con 0x7f3a0010b1a0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80116d0 con 0x7f3a0010b1a0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.445+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a001a7b00 con 0x7f3a0010b1a0 2024-09-10T22:56:53.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.446+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a001051f0 con 0x7f3a0010b1a0 2024-09-10T22:56:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.448+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f39f800f040 con 0x7f3a0010b1a0 2024-09-10T22:56:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.450+0000 7f39fd600640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 0x7f39e80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.450+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f39f80e5e40 con 0x7f3a0010b1a0 2024-09-10T22:56:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.450+0000 7f3a05000640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 0x7f39e80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.451+0000 7f3a05000640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 0x7f39e80b7920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f400abe0 tx=0x7f39f400a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:53.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.454+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f39f8016030 con 0x7f3a0010b1a0 2024-09-10T22:56:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:53 smithi145 ceph-mon[34397]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.665+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3a001a7e60 con 0x7f3a0010b1a0 2024-09-10T22:56:53.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:53 smithi067 ceph-mon[30939]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.665+0000 7f39fd600640 1 -- 172.21.15.67:0/725633946 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+882 (secure 0 0 0) 0x7f39f80ac990 con 0x7f3a0010b1a0 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2682299290 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3212905705 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/82707279 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3042343451 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3255023915 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2935874097 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:53.668 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3802899292 2024-09-11T22:47:16.118298+0000 2024-09-10T22:56:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3004982929 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1673435000 2024-09-11T22:48:31.958130+0000 2024-09-10T22:56:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 msgr2=0x7f39e80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 0x7f39e80b7920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f400abe0 tx=0x7f39f400a5c0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 msgr2=0x7f3a001a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39f800eaa0 tx=0x7f39f800ef60 comp rx=0 tx=0).stop 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 shutdown_connections 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39e80b5460 0x7f39e80b7920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a0010b1a0 0x7f3a001a0980 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a0010a820 0x7f3a0019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 --2- 172.21.15.67:0/725633946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a00104e10 0x7f3a0019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 >> 172.21.15.67:0/725633946 conn(0x7f3a00100840 msgr2=0x7f3a00103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 shutdown_connections 2024-09-10T22:56:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:56:53.668+0000 7f3a06400640 1 -- 172.21.15.67:0/725633946 wait complete. 2024-09-10T22:56:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:53 smithi092 ceph-mon[34223]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:54.425 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-09-10T22:56:54.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:56:54.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:56:54.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[78848]: fuse finished with error 0 and tester_r 0 2024-09-10T22:56:54.484 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:56:54.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:54 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/725633946' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:54.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:54 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/725633946' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:54.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:54 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/725633946' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:56:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:55 smithi145 ceph-mon[34397]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:55 smithi092 ceph-mon[34223]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:55 smithi067 ceph-mon[30939]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:56:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:57 smithi067 ceph-mon[30939]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:56:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:57 smithi145 ceph-mon[34397]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:56:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:57 smithi092 ceph-mon[34223]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:56:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:56:59 smithi145 ceph-mon[34397]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:56:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:56:59 smithi092 ceph-mon[34223]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:56:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:56:59 smithi067 ceph-mon[30939]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:00.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:57:00.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:57:00.586 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T22:57:00.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:57:01.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:01.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:01 smithi145 ceph-mon[34397]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:01.745 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:01 smithi067 ceph-mon[30939]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:01 smithi092 ceph-mon[34223]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.857+0000 7faaa9e00640 1 -- 172.21.15.67:0/931889959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4109150 msgr2=0x7faaa410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.857+0000 7faaa9e00640 1 --2- 172.21.15.67:0/931889959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4109150 0x7faaa410f950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009ca0 tx=0x7faa9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 -- 172.21.15.67:0/931889959 shutdown_connections 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 --2- 172.21.15.67:0/931889959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4109150 0x7faaa410f950 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 --2- 172.21.15.67:0/931889959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa41087d0 0x7faaa4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 --2- 172.21.15.67:0/931889959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4102dc0 0x7faaa41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 -- 172.21.15.67:0/931889959 >> 172.21.15.67:0/931889959 conn(0x7faaa40fe830 msgr2=0x7faaa4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 -- 172.21.15.67:0/931889959 shutdown_connections 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.858+0000 7faaa9e00640 1 -- 172.21.15.67:0/931889959 wait complete. 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 Processor -- start 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 -- start start 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa41087d0 0x7faaa419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa4109150 0x7faaa41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4112d10 con 0x7faaa4102dc0 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4112b90 con 0x7faaa4109150 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4112e90 con 0x7faaa41087d0 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35670/0 (socket says 172.21.15.67:35670) 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa2a00640 1 -- 172.21.15.67:0/2023970472 learned_addr learned my addr 172.21.15.67:0/2023970472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.859+0000 7faaa2000640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa41087d0 0x7faaa419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa3400640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa4109150 0x7faaa41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 -- 172.21.15.67:0/2023970472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa41087d0 msgr2=0x7faaa419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa41087d0 0x7faaa419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 -- 172.21.15.67:0/2023970472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa4109150 msgr2=0x7faaa41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa4109150 0x7faaa41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 -- 172.21.15.67:0/2023970472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa41a13b0 con 0x7faaa4102dc0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa2a00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faa9400c970 tx=0x7faa9400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94007bb0 con 0x7faaa4102dc0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa94007d50 con 0x7faaa4102dc0 2024-09-10T22:57:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.860+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94004d80 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.861+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa41a16a0 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.861+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa41a7d60 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.862+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa41031a0 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.864+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7faa94004f20 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.865+0000 7faaa8a00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 0x7faa880b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.866+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7faa940e5fa0 con 0x7faaa4102dc0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.866+0000 7faaa2000640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 0x7faa880b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.866+0000 7faaa2000640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 0x7faa880b7870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faaa4108370 tx=0x7faa9000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:01.867+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faa94014030 con 0x7faaa4102dc0 2024-09-10T22:57:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.063+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaa402e5c0 con 0x7faaa4102dc0 2024-09-10T22:57:02.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.065+0000 7faaa8a00640 1 -- 172.21.15.67:0/2023970472 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7faa940acb70 con 0x7faaa4102dc0 2024-09-10T22:57:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:57:02.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 msgr2=0x7faa880b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 0x7faa880b7870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faaa4108370 tx=0x7faa9000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 msgr2=0x7faaa419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faa9400c970 tx=0x7faa9400cd70 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 shutdown_connections 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa880b53b0 0x7faa880b7870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa4109150 0x7faaa41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa41087d0 0x7faaa419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 --2- 172.21.15.67:0/2023970472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa4102dc0 0x7faaa419b300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 >> 172.21.15.67:0/2023970472 conn(0x7faaa40fe830 msgr2=0x7faaa4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 shutdown_connections 2024-09-10T22:57:02.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:02.068+0000 7faaa9e00640 1 -- 172.21.15.67:0/2023970472 wait complete. 2024-09-10T22:57:02.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:02 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2023970472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:02.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:02 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2023970472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:02.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:02 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2023970472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:02.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:57:03.327 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:03 smithi145 ceph-mon[34397]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:03.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:03 smithi067 ceph-mon[30939]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:03 smithi092 ceph-mon[34223]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:57:03.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 -- 172.21.15.67:0/343436137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc1087d0 msgr2=0x7f7dbc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 --2- 172.21.15.67:0/343436137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc108c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009860 tx=0x7f7dac02a340 comp rx=0 tx=0).stop 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 -- 172.21.15.67:0/343436137 shutdown_connections 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 --2- 172.21.15.67:0/343436137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc109150 0x7f7dbc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 --2- 172.21.15.67:0/343436137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc108c10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 --2- 172.21.15.67:0/343436137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 -- 172.21.15.67:0/343436137 >> 172.21.15.67:0/343436137 conn(0x7f7dbc0fe830 msgr2=0x7f7dbc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 -- 172.21.15.67:0/343436137 shutdown_connections 2024-09-10T22:57:03.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.934+0000 7f7dc4800640 1 -- 172.21.15.67:0/343436137 wait complete. 2024-09-10T22:57:03.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.935+0000 7f7dc4800640 1 Processor -- start 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 -- start start 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc109150 0x7f7dbc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc112cc0 con 0x7f7dbc1087d0 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc112b40 con 0x7f7dbc102dc0 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc4800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc112e40 con 0x7f7dbc109150 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35688/0 (socket says 172.21.15.67:35688) 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.936+0000 7f7dc1400640 1 -- 172.21.15.67:0/800642179 learned_addr learned my addr 172.21.15.67:0/800642179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1e00640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc2800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc109150 0x7f7dbc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 -- 172.21.15.67:0/800642179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc109150 msgr2=0x7f7dbc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc109150 0x7f7dbc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 -- 172.21.15.67:0/800642179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 msgr2=0x7f7dbc1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 -- 172.21.15.67:0/800642179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19f2a0 con 0x7f7dbc1087d0 2024-09-10T22:57:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1e00640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.937+0000 7f7dc1400640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0057c0 tx=0x7f7dac02ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.938+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac040020 con 0x7f7dbc1087d0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.938+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dac0044d0 con 0x7f7dbc1087d0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.938+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc19f530 con 0x7f7dbc1087d0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.938+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac037040 con 0x7f7dbc1087d0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.938+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1a5c50 con 0x7f7dbc1087d0 2024-09-10T22:57:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.939+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc1031a0 con 0x7f7dbc1087d0 2024-09-10T22:57:03.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.941+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f7dac0303d0 con 0x7f7dbc1087d0 2024-09-10T22:57:03.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.942+0000 7f7db6a00640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 0x7f7da40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.943+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f7dac106190 con 0x7f7dbc1087d0 2024-09-10T22:57:03.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.943+0000 7f7dc1e00640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 0x7f7da40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.943+0000 7f7dc1e00640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 0x7f7da40b77f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fc0 tx=0x7f7db0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:03.947+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7dac04a020 con 0x7f7dbc1087d0 2024-09-10T22:57:04.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.144+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dbc10a9a0 con 0x7f7dbc1087d0 2024-09-10T22:57:04.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.145+0000 7f7db6a00640 1 -- 172.21.15.67:0/800642179 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7f7dac0ccce0 con 0x7f7dbc1087d0 2024-09-10T22:57:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:04.148 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 msgr2=0x7f7da40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 0x7f7da40b77f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fc0 tx=0x7f7db0005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 msgr2=0x7f7dbc199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0057c0 tx=0x7f7dac02ae60 comp rx=0 tx=0).stop 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 shutdown_connections 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7da40b5330 0x7f7da40b77f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dbc109150 0x7f7dbc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dbc1087d0 0x7f7dbc199730 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 --2- 172.21.15.67:0/800642179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dbc102dc0 0x7f7dbc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 >> 172.21.15.67:0/800642179 conn(0x7f7dbc0fe830 msgr2=0x7f7dbc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 shutdown_connections 2024-09-10T22:57:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:04.148+0000 7f7dc4800640 1 -- 172.21.15.67:0/800642179 wait complete. 2024-09-10T22:57:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:04 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/800642179' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:04.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:04 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/800642179' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:04.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:04 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/800642179' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:04.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:57:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:05 smithi145 ceph-mon[34397]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:05.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:05 smithi067 ceph-mon[30939]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:05 smithi092 ceph-mon[34223]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.832+0000 7f8976c00640 1 -- 172.21.15.67:0/454648432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 msgr2=0x7f8970103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.832+0000 7f8976c00640 1 --2- 172.21.15.67:0/454648432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f8970103180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8960009900 tx=0x7f896002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 -- 172.21.15.67:0/454648432 shutdown_connections 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 --2- 172.21.15.67:0/454648432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 0x7f897010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 --2- 172.21.15.67:0/454648432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 0x7f8970108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 --2- 172.21.15.67:0/454648432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f8970103180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 -- 172.21.15.67:0/454648432 >> 172.21.15.67:0/454648432 conn(0x7f89700fe810 msgr2=0x7f8970100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 -- 172.21.15.67:0/454648432 shutdown_connections 2024-09-10T22:57:05.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.833+0000 7f8976c00640 1 -- 172.21.15.67:0/454648432 wait complete. 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 Processor -- start 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 -- start start 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 0x7f897019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 0x7f89701a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112cf0 con 0x7f8970102da0 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112b70 con 0x7f8970109130 2024-09-10T22:57:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f8976c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8970112e70 con 0x7f89701087b0 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f896fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f896fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35704/0 (socket says 172.21.15.67:35704) 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.834+0000 7f896fe00640 1 -- 172.21.15.67:0/2986663206 learned_addr learned my addr 172.21.15.67:0/2986663206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f8974c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 0x7f89701a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f8967400640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 0x7f897019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 -- 172.21.15.67:0/2986663206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 msgr2=0x7f897019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 0x7f897019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 -- 172.21.15.67:0/2986663206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 msgr2=0x7f89701a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 0x7f89701a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 -- 172.21.15.67:0/2986663206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701a1460 con 0x7f8970102da0 2024-09-10T22:57:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.835+0000 7f896fe00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f896002a910 tx=0x7f8960003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.836+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896003f070 con 0x7f8970102da0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.836+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f896003bdc0 con 0x7f8970102da0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.836+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89701a16f0 con 0x7f8970102da0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.836+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89701a7e10 con 0x7f8970102da0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.836+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960035030 con 0x7f8970102da0 2024-09-10T22:57:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.837+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8970103180 con 0x7f8970102da0 2024-09-10T22:57:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.841+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8960031500 con 0x7f8970102da0 2024-09-10T22:57:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.842+0000 7f896d600640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 0x7f89540b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.842+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f8960106fd0 con 0x7f8970102da0 2024-09-10T22:57:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.842+0000 7f8967400640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 0x7f89540b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.843+0000 7f8967400640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 0x7f89540b77f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8970108550 tx=0x7f895800d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:05.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:05.844+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f89600cd5c0 con 0x7f8970102da0 2024-09-10T22:57:06.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.080+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f897010f320 con 0x7f8970102da0 2024-09-10T22:57:06.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.378+0000 7f896d600640 1 -- 172.21.15.67:0/2986663206 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f89600d1ff0 con 0x7f8970102da0 2024-09-10T22:57:06.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 msgr2=0x7f89540b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:06.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 0x7f89540b77f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8970108550 tx=0x7f895800d040 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 msgr2=0x7f897019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f896002a910 tx=0x7f8960003d60 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 shutdown_connections 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f89540b5330 0x7f89540b77f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8970109130 0x7f89701a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89701087b0 0x7f897019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 --2- 172.21.15.67:0/2986663206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8970102da0 0x7f897019b3b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:06.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.381+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 >> 172.21.15.67:0/2986663206 conn(0x7f89700fe810 msgr2=0x7f8970104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.382+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 shutdown_connections 2024-09-10T22:57:06.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:06.382+0000 7f8976c00640 1 -- 172.21.15.67:0/2986663206 wait complete. 2024-09-10T22:57:06.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:06 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:57:06.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:06 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:57:06.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:06 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:57:07.205 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T22:57:07.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-10T22:57:07.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:07 smithi067 ceph-mon[30939]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:07.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:07 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:57:07.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:07 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:07.611 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:07 smithi092 ceph-mon[34223]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:07 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:57:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:07 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:07 smithi145 ceph-mon[34397]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:07 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2986663206' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:57:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:07 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 -- 172.21.15.67:0/1465530967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71410f390 msgr2=0x7fe714111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 --2- 172.21.15.67:0/1465530967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71410f390 0x7fe714111780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009ca0 tx=0x7fe70c02a8b0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 -- 172.21.15.67:0/1465530967 shutdown_connections 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 --2- 172.21.15.67:0/1465530967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71410f390 0x7fe714111780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 --2- 172.21.15.67:0/1465530967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe714075e00 0x7fe71410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 --2- 172.21.15.67:0/1465530967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7140754e0 0x7fe7140758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 -- 172.21.15.67:0/1465530967 >> 172.21.15.67:0/1465530967 conn(0x7fe7140fe810 msgr2=0x7fe714100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 -- 172.21.15.67:0/1465530967 shutdown_connections 2024-09-10T22:57:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.135+0000 7fe71a200640 1 -- 172.21.15.67:0/1465530967 wait complete. 2024-09-10T22:57:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.136+0000 7fe71a200640 1 Processor -- start 2024-09-10T22:57:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.136+0000 7fe71a200640 1 -- start start 2024-09-10T22:57:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 0x7fe71419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71410f390 0x7fe7141a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe714114e10 con 0x7fe7140754e0 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe714114c90 con 0x7fe714075e00 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe71a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe714114f90 con 0x7fe71410f390 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe713400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe713400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35728/0 (socket says 172.21.15.67:35728) 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe713400640 1 -- 172.21.15.67:0/4285567684 learned_addr learned my addr 172.21.15.67:0/4285567684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe70aa00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 0x7fe71419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.137+0000 7fe713e00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71410f390 0x7fe7141a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 -- 172.21.15.67:0/4285567684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71410f390 msgr2=0x7fe7141a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71410f390 0x7fe7141a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 -- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 msgr2=0x7fe71419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 -- 172.21.15.67:0/4285567684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141a1410 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe713400640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe70aa00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 0x7fe71419b8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d720 tx=0x7fe6fc00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.138+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc017070 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.139+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6fc00de30 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.139+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a1700 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.139+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a7dc0 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.139+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0047a0 con 0x7fe714075e00 2024-09-10T22:57:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.140+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f00052c0 con 0x7fe714075e00 2024-09-10T22:57:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.142+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe6fc004220 con 0x7fe714075e00 2024-09-10T22:57:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.144+0000 7fe710c00640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 0x7fe6f80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.144+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fe6fc0e5d30 con 0x7fe714075e00 2024-09-10T22:57:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.144+0000 7fe713400640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 0x7fe6f80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.145+0000 7fe713400640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 0x7fe6f80b7870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe704005fd0 tx=0x7fe70400b4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.146+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6fc020020 con 0x7fe714075e00 2024-09-10T22:57:08.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:08.389+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fe6f0005880 con 0x7fe714075e00 2024-09-10T22:57:09.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.396+0000 7fe710c00640 1 -- 172.21.15.67:0/4285567684 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fe6fc0ac900 con 0x7fe714075e00 2024-09-10T22:57:09.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 msgr2=0x7fe6f80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:09.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 0x7fe6f80b7870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe704005fd0 tx=0x7fe70400b4a0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 msgr2=0x7fe71419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 0x7fe71419b8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d720 tx=0x7fe6fc00dbe0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 shutdown_connections 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6f80b53b0 0x7fe6f80b7870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71410f390 0x7fe7141a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe714075e00 0x7fe71419b8a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 --2- 172.21.15.67:0/4285567684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7140754e0 0x7fe71419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.399+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 >> 172.21.15.67:0/4285567684 conn(0x7fe7140fe810 msgr2=0x7fe7140ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.400+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 shutdown_connections 2024-09-10T22:57:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:09.400+0000 7fe71a200640 1 -- 172.21.15.67:0/4285567684 wait complete. 2024-09-10T22:57:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:09 smithi092 ceph-mon[34223]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:57:09.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:09 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4285567684' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:09.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:09 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:09 smithi067 ceph-mon[30939]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:57:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:09 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4285567684' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:09 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:09 smithi145 ceph-mon[34397]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:57:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:09 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4285567684' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:09.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:09 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-10T22:57:10.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T22:57:10.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:57:10.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T22:57:10.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:57:10.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:10.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:57:10.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:10 smithi067 ceph-mon[30939]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:57:10.679 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:10.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:57:10.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T22:57:10.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:57:10.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:10.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:57:10.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:10 smithi092 ceph-mon[34223]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:57:10.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:57:10.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T22:57:10.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:57:10.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:57:10.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:57:10.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:10 smithi145 ceph-mon[34397]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:57:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 -- 172.21.15.67:0/3161387906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff200102da0 msgr2=0x7ff200103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 --2- 172.21.15.67:0/3161387906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff200102da0 0x7ff200103180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009900 tx=0x7ff1f002a410 comp rx=0 tx=0).stop 2024-09-10T22:57:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 -- 172.21.15.67:0/3161387906 shutdown_connections 2024-09-10T22:57:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 --2- 172.21.15.67:0/3161387906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200109130 0x7ff20010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 --2- 172.21.15.67:0/3161387906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2001087b0 0x7ff200108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 --2- 172.21.15.67:0/3161387906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff200102da0 0x7ff200103180 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 -- 172.21.15.67:0/3161387906 >> 172.21.15.67:0/3161387906 conn(0x7ff2000fe810 msgr2=0x7ff200100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 -- 172.21.15.67:0/3161387906 shutdown_connections 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.307+0000 7ff206200640 1 -- 172.21.15.67:0/3161387906 wait complete. 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.308+0000 7ff206200640 1 Processor -- start 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 -- start start 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200102da0 0x7ff2000754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 0x7ff200075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff200112cf0 con 0x7ff2001087b0 2024-09-10T22:57:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff200112b70 con 0x7ff200109130 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.309+0000 7ff206200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff200112e70 con 0x7ff200102da0 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1ff400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200102da0 0x7ff2000754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56126/0 (socket says 172.21.15.67:56126) 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 -- 172.21.15.67:0/4262653851 learned_addr learned my addr 172.21.15.67:0/4262653851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1ffe00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 0x7ff200075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 -- 172.21.15.67:0/4262653851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200102da0 msgr2=0x7ff2000754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200102da0 0x7ff2000754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 -- 172.21.15.67:0/4262653851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 msgr2=0x7ff200075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 0x7ff200075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 -- 172.21.15.67:0/4262653851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2001a75c0 con 0x7ff2001087b0 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1ffe00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 0x7ff200075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1f6a00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.310+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800cd80 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.311+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e80108c0 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.311+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80187c0 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.311+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2001a78b0 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.311+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2001a7da0 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.312+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff200103180 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.314+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7ff1e8010430 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.315+0000 7ff1fcc00640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 0x7ff1e40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.316+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7ff1e80e67e0 con 0x7ff2001087b0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.316+0000 7ff1ff400640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 0x7ff1e40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.316+0000 7ff1ff400640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 0x7ff1e40b7870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0006010 tx=0x7ff1f0005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.317+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1e80ad330 con 0x7ff2001087b0 2024-09-10T22:57:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:active 2024-09-10T22:57:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:57:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:57:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T22:57:11.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:57:11.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:creating} 9 up:standby 2024-09-10T22:57:11.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:11 smithi067 ceph-mon[30939]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-09-10T22:57:11.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:11.556+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff2000767a0 con 0x7ff2001087b0 2024-09-10T22:57:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:active 2024-09-10T22:57:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:57:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:57:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T22:57:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:57:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:creating} 9 up:standby 2024-09-10T22:57:11.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:11 smithi092 ceph-mon[34223]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-09-10T22:57:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:57:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] up:active 2024-09-10T22:57:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:57:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:57:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T22:57:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:57:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:creating} 9 up:standby 2024-09-10T22:57:11.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:11 smithi145 ceph-mon[34397]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-09-10T22:57:12.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.420+0000 7ff1fcc00640 1 -- 172.21.15.67:0/4262653851 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7ff2000767a0 con 0x7ff2001087b0 2024-09-10T22:57:12.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.422+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 msgr2=0x7ff1e40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.422+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 0x7ff1e40b7870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0006010 tx=0x7ff1f0005e80 comp rx=0 tx=0).stop 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.422+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 msgr2=0x7ff2000759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.422+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 shutdown_connections 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53b0 0x7ff1e40b7870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:12.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff200109130 0x7ff200075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:12.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2001087b0 0x7ff2000759e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:12.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 --2- 172.21.15.67:0/4262653851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff200102da0 0x7ff2000754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:12.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 >> 172.21.15.67:0/4262653851 conn(0x7ff2000fe810 msgr2=0x7ff200100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:12.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 shutdown_connections 2024-09-10T22:57:12.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:12.423+0000 7ff206200640 1 -- 172.21.15.67:0/4262653851 wait complete. 2024-09-10T22:57:12.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:12 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:active 2024-09-10T22:57:12.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:12 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:12.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:12 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:57:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:12 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:active 2024-09-10T22:57:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:12 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:12 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:57:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:12 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] up:active 2024-09-10T22:57:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:12 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:12.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:12 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:57:13.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T22:57:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:13.771 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 22:57:13 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: 2024-09-10T22:57:13.377+0000 7f544aa00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T22:57:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:13 smithi092 ceph-mon[34223]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 2 op/s 2024-09-10T22:57:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:57:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:13 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:13.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:13 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:13 smithi067 ceph-mon[30939]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 2 op/s 2024-09-10T22:57:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:13 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:57:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:13 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:13 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:13 smithi145 ceph-mon[34397]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 2 op/s 2024-09-10T22:57:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:13 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4262653851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:57:13.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:13 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 9 up:standby 2024-09-10T22:57:13.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:13 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 -- 172.21.15.67:0/626590128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c102da0 msgr2=0x7f900c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 --2- 172.21.15.67:0/626590128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c102da0 0x7f900c103180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009900 tx=0x7f8ffc02a410 comp rx=0 tx=0).stop 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 -- 172.21.15.67:0/626590128 shutdown_connections 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 --2- 172.21.15.67:0/626590128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c109130 0x7f900c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 --2- 172.21.15.67:0/626590128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c1087b0 0x7f900c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 --2- 172.21.15.67:0/626590128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c102da0 0x7f900c103180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:14.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 -- 172.21.15.67:0/626590128 >> 172.21.15.67:0/626590128 conn(0x7f900c0fe810 msgr2=0x7f900c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:14.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 -- 172.21.15.67:0/626590128 shutdown_connections 2024-09-10T22:57:14.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.324+0000 7f9013400640 1 -- 172.21.15.67:0/626590128 wait complete. 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.325+0000 7f9013400640 1 Processor -- start 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.325+0000 7f9013400640 1 -- start start 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c1087b0 0x7f900c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f900c112cf0 con 0x7f900c109130 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f900c112b70 con 0x7f900c1087b0 2024-09-10T22:57:14.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9013400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f900c112e70 con 0x7f900c102da0 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9011400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9010a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9011400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56150/0 (socket says 172.21.15.67:56150) 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9010a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:60660/0 (socket says 172.21.15.67:60660) 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9011400640 1 -- 172.21.15.67:0/2234363254 learned_addr learned my addr 172.21.15.67:0/2234363254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.326+0000 7f9003e00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c1087b0 0x7f900c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 -- 172.21.15.67:0/2234363254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c1087b0 msgr2=0x7f900c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c1087b0 0x7f900c19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 -- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 msgr2=0x7f900c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 -- 172.21.15.67:0/2234363254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1a1520 con 0x7f900c102da0 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9011400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.327+0000 7f9010a00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc02a930 tx=0x7f8ffc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.328+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc03f070 con 0x7f900c102da0 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.328+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c1a17b0 con 0x7f900c102da0 2024-09-10T22:57:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.328+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ffc03baf0 con 0x7f900c102da0 2024-09-10T22:57:14.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.328+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc035030 con 0x7f900c102da0 2024-09-10T22:57:14.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.328+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a7ed0 con 0x7f900c102da0 2024-09-10T22:57:14.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.329+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c103180 con 0x7f900c102da0 2024-09-10T22:57:14.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.331+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8ffc03b660 con 0x7f900c102da0 2024-09-10T22:57:14.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.333+0000 7f900a000640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 0x7f8ff00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:14.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.333+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f8ffc106de0 con 0x7f900c102da0 2024-09-10T22:57:14.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.333+0000 7f9003e00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 0x7f8ff00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:14.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.334+0000 7f9003e00640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 0x7f8ff00b77f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f900c101dd0 tx=0x7f8ff4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:14.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.335+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8ffc0cd2e0 con 0x7f900c102da0 2024-09-10T22:57:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:14.571+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f900c19c5a0 con 0x7f900c102da0 2024-09-10T22:57:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.429+0000 7f900a000640 1 -- 172.21.15.67:0/2234363254 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f8ffc0d1d10 con 0x7f900c102da0 2024-09-10T22:57:15.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.432+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 msgr2=0x7f8ff00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.432+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 0x7f8ff00b77f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f900c101dd0 tx=0x7f8ff4005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.432+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 msgr2=0x7f900c19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.432+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc02a930 tx=0x7f8ffc003b90 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 shutdown_connections 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8ff00b5330 0x7f8ff00b77f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f900c109130 0x7f900c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f900c1087b0 0x7f900c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 --2- 172.21.15.67:0/2234363254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c102da0 0x7f900c19b470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 >> 172.21.15.67:0/2234363254 conn(0x7f900c0fe810 msgr2=0x7f900c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 shutdown_connections 2024-09-10T22:57:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:15.433+0000 7f9013400640 1 -- 172.21.15.67:0/2234363254 wait complete. 2024-09-10T22:57:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:15 smithi092 ceph-mon[34223]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:57:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:15 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2234363254' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:15 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:15 smithi067 ceph-mon[30939]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:57:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:15 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2234363254' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:15 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:15 smithi145 ceph-mon[34397]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:57:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:15 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2234363254' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:15 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:57:16.401 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-10T22:57:16.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T22:57:16.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:16 smithi092 ceph-mon[34223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:57:16.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:16 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:16.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:16 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:16.790 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:16.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:16 smithi067 ceph-mon[30939]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:57:16.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:16 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:16.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:16 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:16.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:16 smithi145 ceph-mon[34397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:57:16.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:16 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:57:16.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:16 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 -- 172.21.15.67:0/2063985226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04102c10 msgr2=0x7f6f04102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2063985226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04102c10 0x7f6f04102ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009900 tx=0x7f6ef402a410 comp rx=0 tx=0).stop 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 -- 172.21.15.67:0/2063985226 shutdown_connections 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2063985226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108fa0 0x7f6f0410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2063985226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04108620 0x7f6f04108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2063985226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04102c10 0x7f6f04102ff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 -- 172.21.15.67:0/2063985226 >> 172.21.15.67:0/2063985226 conn(0x7f6f040fe660 msgr2=0x7f6f04100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 -- 172.21.15.67:0/2063985226 shutdown_connections 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.365+0000 7f6f03e00640 1 -- 172.21.15.67:0/2063985226 wait complete. 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.366+0000 7f6f03e00640 1 Processor -- start 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 -- start start 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04102c10 0x7f6f04198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 0x7f6f041994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04112b60 con 0x7f6f04108fa0 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041129e0 con 0x7f6f04108620 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04112ce0 con 0x7f6f04102c10 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56168/0 (socket says 172.21.15.67:56168) 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f03400640 1 -- 172.21.15.67:0/876777238 learned_addr learned my addr 172.21.15.67:0/876777238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f02000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 0x7f6f041994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.367+0000 7f6f02a00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04102c10 0x7f6f04198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 -- 172.21.15.67:0/876777238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04102c10 msgr2=0x7f6f04198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04102c10 0x7f6f04198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 -- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 msgr2=0x7f6f0419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 -- 172.21.15.67:0/876777238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0419f020 con 0x7f6f04108620 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f03400640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f02000640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 0x7f6f041994b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000d720 tx=0x7f6ef000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0017070 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.368+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0419f310 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.369+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ef000de30 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.369+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041a59d0 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.369+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00047a0 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.370+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee40052c0 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.373+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6ef0004220 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.374+0000 7f6efb400640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 0x7f6ee80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.374+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f6ef00e59a0 con 0x7f6f04108620 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.374+0000 7f6f02a00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 0x7f6ee80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.375+0000 7f6f02a00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 0x7f6ee80b7920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40057c0 tx=0x7f6ef4005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.379+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6ef0020020 con 0x7f6f04108620 2024-09-10T22:57:17.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-mon[30939]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:17.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.621+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6ee4000bf0 con 0x7f6ee80b5460 2024-09-10T22:57:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.653+0000 7f6efb400640 1 -- 172.21.15.67:0/876777238 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ee4000bf0 con 0x7f6ee80b5460 2024-09-10T22:57:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 msgr2=0x7f6ee80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 0x7f6ee80b7920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40057c0 tx=0x7f6ef4005a20 comp rx=0 tx=0).stop 2024-09-10T22:57:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 msgr2=0x7f6f041994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 0x7f6f041994b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000d720 tx=0x7f6ef000dbe0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 shutdown_connections 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6ee80b5460 0x7f6ee80b7920 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f04108fa0 0x7f6f0419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f04108620 0x7f6f041994b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 --2- 172.21.15.67:0/876777238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f04102c10 0x7f6f04198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.655+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 >> 172.21.15.67:0/876777238 conn(0x7f6f040fe660 msgr2=0x7f6f04100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.656+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 shutdown_connections 2024-09-10T22:57:17.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:17.656+0000 7f6f03e00640 1 -- 172.21.15.67:0/876777238 wait complete. 2024-09-10T22:57:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:17 smithi092 ceph-mon[34223]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:17 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:17 smithi145 ceph-mon[34397]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:17 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:17.899 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:57:17.631+0000 7fca0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:57:17.900 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:57:17.631+0000 7fca0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:57:17.900 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:57:17.631+0000 7fca0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:57:17.900 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:57:17.631+0000 7fca0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:57:17.900 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 22:57:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T22:57:17.631+0000 7fca0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:57:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:18 smithi067 ceph-mon[30939]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:18 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:18 smithi092 ceph-mon[34223]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:18.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:18 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:18 smithi145 ceph-mon[34397]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:18.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:18.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:18 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:19.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T22:57:19.612 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:19.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:19 smithi067 ceph-mon[30939]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:19.753 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:19 smithi067 ceph-mon[30939]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-10T22:57:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:19 smithi092 ceph-mon[34223]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:19 smithi092 ceph-mon[34223]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-10T22:57:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:19 smithi145 ceph-mon[34397]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T22:57:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:19 smithi145 ceph-mon[34397]: mgrmap e21: x(active, since 8m), standbys: z, y 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.182+0000 7fb043e00640 1 -- 172.21.15.67:0/187778364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 msgr2=0x7fb04410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.182+0000 7fb043e00640 1 --2- 172.21.15.67:0/187778364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04410ac40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009900 tx=0x7fb02c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 -- 172.21.15.67:0/187778364 shutdown_connections 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 --2- 172.21.15.67:0/187778364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb04410b180 0x7fb044111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 --2- 172.21.15.67:0/187778364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04410ac40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 --2- 172.21.15.67:0/187778364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044104df0 0x7fb0441051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 -- 172.21.15.67:0/187778364 >> 172.21.15.67:0/187778364 conn(0x7fb044100840 msgr2=0x7fb044102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 -- 172.21.15.67:0/187778364 shutdown_connections 2024-09-10T22:57:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.183+0000 7fb043e00640 1 -- 172.21.15.67:0/187778364 wait complete. 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 Processor -- start 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 -- start start 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044104df0 0x7fb04419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410b180 0x7fb0441a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb044114cf0 con 0x7fb04410a800 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb044114b70 con 0x7fb04410b180 2024-09-10T22:57:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb043e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb044114e70 con 0x7fb044104df0 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb03a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb03a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36512/0 (socket says 172.21.15.67:36512) 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.184+0000 7fb03a000640 1 -- 172.21.15.67:0/1498609214 learned_addr learned my addr 172.21.15.67:0/1498609214 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb042a00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044104df0 0x7fb04419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb043400640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410b180 0x7fb0441a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 -- 172.21.15.67:0/1498609214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044104df0 msgr2=0x7fb04419b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044104df0 0x7fb04419b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 -- 172.21.15.67:0/1498609214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410b180 msgr2=0x7fb0441a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410b180 0x7fb0441a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 -- 172.21.15.67:0/1498609214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0441a1150 con 0x7fb04410a800 2024-09-10T22:57:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.185+0000 7fb03a000640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009230 tx=0x7fb02c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.186+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c03f030 con 0x7fb04410a800 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.186+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb02c003a80 con 0x7fb04410a800 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.186+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c035070 con 0x7fb04410a800 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.186+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441a13e0 con 0x7fb04410a800 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.186+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a7b00 con 0x7fb04410a800 2024-09-10T22:57:20.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.187+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0441051d0 con 0x7fb04410a800 2024-09-10T22:57:20.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.189+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb02c03bc30 con 0x7fb04410a800 2024-09-10T22:57:20.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.190+0000 7fb03be00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 0x7fb0280b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:20.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.191+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fb02c106ac0 con 0x7fb04410a800 2024-09-10T22:57:20.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.191+0000 7fb042a00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 0x7fb0280b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:20.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.191+0000 7fb042a00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 0x7fb0280b7920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb03400bbe0 tx=0x7fb03400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:20.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.194+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb02c0cce60 con 0x7fb04410a800 2024-09-10T22:57:20.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.470+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb044063590 con 0x7fb0280b5460 2024-09-10T22:57:20.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.501+0000 7fb03be00640 1 -- 172.21.15.67:0/1498609214 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb044063590 con 0x7fb0280b5460 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 msgr2=0x7fb0280b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 0x7fb0280b7920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb03400bbe0 tx=0x7fb03400b390 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 msgr2=0x7fb04419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009230 tx=0x7fb02c004270 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 shutdown_connections 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5460 0x7fb0280b7920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410b180 0x7fb0441a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410a800 0x7fb04419b5e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 --2- 172.21.15.67:0/1498609214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044104df0 0x7fb04419b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 >> 172.21.15.67:0/1498609214 conn(0x7fb044100840 msgr2=0x7fb044106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 shutdown_connections 2024-09-10T22:57:20.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:20.504+0000 7fb043e00640 1 -- 172.21.15.67:0/1498609214 wait complete. 2024-09-10T22:57:21.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T22:57:21.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:21.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.724 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T22:57:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:21.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:21.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:21.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:21.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:57:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 -- 172.21.15.67:0/1410235440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0810b1a0 msgr2=0x7f4f081119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 --2- 172.21.15.67:0/1410235440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0810b1a0 0x7f4f081119a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009ca0 tx=0x7f4f0002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 -- 172.21.15.67:0/1410235440 shutdown_connections 2024-09-10T22:57:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 --2- 172.21.15.67:0/1410235440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0810b1a0 0x7f4f081119a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 --2- 172.21.15.67:0/1410235440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810a820 0x7f4f0810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 --2- 172.21.15.67:0/1410235440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f08104e10 0x7f4f081051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.304+0000 7f4f07e00640 1 -- 172.21.15.67:0/1410235440 >> 172.21.15.67:0/1410235440 conn(0x7f4f08100840 msgr2=0x7f4f08102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.305+0000 7f4f07e00640 1 -- 172.21.15.67:0/1410235440 shutdown_connections 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.305+0000 7f4f07e00640 1 -- 172.21.15.67:0/1410235440 wait complete. 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.305+0000 7f4f07e00640 1 Processor -- start 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 -- start start 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 0x7f4f0819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810b1a0 0x7f4f081a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08114d20 con 0x7f4f08104e10 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08114ba0 con 0x7f4f0810a820 2024-09-10T22:57:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f07e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08114ea0 con 0x7f4f0810b1a0 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36544/0 (socket says 172.21.15.67:36544) 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4f06a00640 1 -- 172.21.15.67:0/3699721757 learned_addr learned my addr 172.21.15.67:0/3699721757 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.306+0000 7f4efe000640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 0x7f4f0819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f07400640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810b1a0 0x7f4f081a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 -- 172.21.15.67:0/3699721757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810b1a0 msgr2=0x7f4f081a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810b1a0 0x7f4f081a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 -- 172.21.15.67:0/3699721757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 msgr2=0x7f4f0819b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 0x7f4f0819b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 -- 172.21.15.67:0/3699721757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a11a0 con 0x7f4f08104e10 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4efe000640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 0x7f4f0819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f06a00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8017070 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ef8009c30 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f620 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a1490 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.307+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a7d20 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.309+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f081051f0 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.311+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4ef8010460 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.313+0000 7f4effe00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 0x7f4eec0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.314+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f4ef80e5da0 con 0x7f4f08104e10 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.314+0000 7f4efe000640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 0x7f4eec0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.314+0000 7f4efe000640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 0x7f4eec0b79a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00058c0 tx=0x7f4ef0005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:22.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.317+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4ef80ac790 con 0x7f4f08104e10 2024-09-10T22:57:22.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.543+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4f08063590 con 0x7f4eec0b54e0 2024-09-10T22:57:22.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.550+0000 7f4effe00640 1 -- 172.21.15.67:0/3699721757 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4f08063590 con 0x7f4eec0b54e0 2024-09-10T22:57:22.552 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_0/8149d884-25b6-438e-a232-f31fd87362b4 2024-09-10T22:57:22.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.552+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 msgr2=0x7f4eec0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.552+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 0x7f4eec0b79a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00058c0 tx=0x7f4ef0005850 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 msgr2=0x7f4f0819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 shutdown_connections 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4eec0b54e0 0x7f4eec0b79a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f0810b1a0 0x7f4f081a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f0810a820 0x7f4f0819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 --2- 172.21.15.67:0/3699721757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f08104e10 0x7f4f0819b0f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 >> 172.21.15.67:0/3699721757 conn(0x7f4f08100840 msgr2=0x7f4f08102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 shutdown_connections 2024-09-10T22:57:22.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:22.553+0000 7f4f07e00640 1 -- 172.21.15.67:0/3699721757 wait complete. 2024-09-10T22:57:23.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T22:57:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:23 smithi092 ceph-mon[34223]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:57:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:23 smithi092 ceph-mon[34223]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:23 smithi067 ceph-mon[30939]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:57:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:23 smithi067 ceph-mon[30939]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:23.862 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:23 smithi145 ceph-mon[34397]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:57:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:23 smithi145 ceph-mon[34397]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 -- 172.21.15.67:0/3664149367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 msgr2=0x7f01f4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 --2- 172.21.15.67:0/3664149367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f4077ab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009900 tx=0x7f01e802a410 comp rx=0 tx=0).stop 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 -- 172.21.15.67:0/3664149367 shutdown_connections 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 --2- 172.21.15.67:0/3664149367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 0x7f01f4115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 --2- 172.21.15.67:0/3664149367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f4113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 --2- 172.21.15.67:0/3664149367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f4077ab0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 -- 172.21.15.67:0/3664149367 >> 172.21.15.67:0/3664149367 conn(0x7f01f4100a50 msgr2=0x7f01f4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 -- 172.21.15.67:0/3664149367 shutdown_connections 2024-09-10T22:57:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.455+0000 7f01fb600640 1 -- 172.21.15.67:0/3664149367 wait complete. 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.456+0000 7f01fb600640 1 Processor -- start 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.456+0000 7f01fb600640 1 -- start start 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 0x7f01f41a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4119230 con 0x7f01f40776d0 2024-09-10T22:57:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41190b0 con 0x7f01f4113990 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01fb600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41193b0 con 0x7f01f4077ff0 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01f3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01f8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01f3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:49974/0 (socket says 172.21.15.67:49974) 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01f3e00640 1 -- 172.21.15.67:0/3770865506 learned_addr learned my addr 172.21.15.67:0/3770865506 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.457+0000 7f01f9600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 0x7f01f41a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 -- 172.21.15.67:0/3770865506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 msgr2=0x7f01f419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 -- 172.21.15.67:0/3770865506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 msgr2=0x7f01f41a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 0x7f01f41a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 -- 172.21.15.67:0/3770865506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f41a5850 con 0x7f01f40776d0 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f8c00640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f419f7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005940 tx=0x7f01e80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e803f070 con 0x7f01f40776d0 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01e8003c80 con 0x7f01f40776d0 2024-09-10T22:57:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.458+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8035070 con 0x7f01f40776d0 2024-09-10T22:57:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.459+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f41a5ae0 con 0x7f01f40776d0 2024-09-10T22:57:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.459+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41ac200 con 0x7f01f40776d0 2024-09-10T22:57:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.460+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f4077ab0 con 0x7f01f40776d0 2024-09-10T22:57:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.462+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f01e804b060 con 0x7f01f40776d0 2024-09-10T22:57:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.463+0000 7f01f1600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 0x7f01dc0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.464+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f01e81063b0 con 0x7f01f40776d0 2024-09-10T22:57:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.464+0000 7f01f3e00640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 0x7f01dc0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.464+0000 7f01f3e00640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 0x7f01dc0b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01f4116110 tx=0x7f01e000b4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.468+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01e80cc7b0 con 0x7f01f40776d0 2024-09-10T22:57:24.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.692+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f01f410eb50 con 0x7f01dc0b5440 2024-09-10T22:57:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.720+0000 7f01f1600640 1 -- 172.21.15.67:0/3770865506 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01f410eb50 con 0x7f01dc0b5440 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 msgr2=0x7f01dc0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 0x7f01dc0b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01f4116110 tx=0x7f01e000b4a0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 msgr2=0x7f01f419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f419f7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005940 tx=0x7f01e80040e0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 shutdown_connections 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01dc0b5440 0x7f01dc0b7900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f4113990 0x7f01f41a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01f4077ff0 0x7f01f419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 --2- 172.21.15.67:0/3770865506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f40776d0 0x7f01f419f7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 >> 172.21.15.67:0/3770865506 conn(0x7f01f4100a50 msgr2=0x7f01f4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 shutdown_connections 2024-09-10T22:57:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:24.723+0000 7f01fb600640 1 -- 172.21.15.67:0/3770865506 wait complete. 2024-09-10T22:57:25.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T22:57:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:25 smithi092 ceph-mon[34223]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:57:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:25 smithi092 ceph-mon[34223]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:25 smithi067 ceph-mon[30939]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:57:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:25 smithi067 ceph-mon[30939]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:25 smithi145 ceph-mon[34397]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:57:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:25 smithi145 ceph-mon[34397]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:25.977 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 -- 172.21.15.67:0/3050166281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484075e00 msgr2=0x7f348410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 --2- 172.21.15.67:0/3050166281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484075e00 0x7f348410ee70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f346c009900 tx=0x7f346c02a410 comp rx=0 tx=0).stop 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 -- 172.21.15.67:0/3050166281 shutdown_connections 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 --2- 172.21.15.67:0/3050166281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348410f3b0 0x7f34841117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 --2- 172.21.15.67:0/3050166281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3484075e00 0x7f348410ee70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 --2- 172.21.15.67:0/3050166281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34840754e0 0x7f34840758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 -- 172.21.15.67:0/3050166281 >> 172.21.15.67:0/3050166281 conn(0x7f34840fe830 msgr2=0x7f3484100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:26.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 -- 172.21.15.67:0/3050166281 shutdown_connections 2024-09-10T22:57:26.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.605+0000 7f348a200640 1 -- 172.21.15.67:0/3050166281 wait complete. 2024-09-10T22:57:26.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 Processor -- start 2024-09-10T22:57:26.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 -- start start 2024-09-10T22:57:26.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34840754e0 0x7f348419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:26.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3484075e00 0x7f348419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3484114e70 con 0x7f348410f3b0 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3484114cf0 con 0x7f34840754e0 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.606+0000 7f348a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3484114ff0 con 0x7f3484075e00 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36572/0 (socket says 172.21.15.67:36572) 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 -- 172.21.15.67:0/2511163059 learned_addr learned my addr 172.21.15.67:0/2511163059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f347aa00640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3484075e00 0x7f348419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483400640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34840754e0 0x7f348419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 -- 172.21.15.67:0/2511163059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3484075e00 msgr2=0x7f348419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3484075e00 0x7f348419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 -- 172.21.15.67:0/2511163059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34840754e0 msgr2=0x7f348419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34840754e0 0x7f348419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.607+0000 7f3483e00640 1 -- 172.21.15.67:0/2511163059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841a1470 con 0x7f348410f3b0 2024-09-10T22:57:26.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f3483e00640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f347c00eb40 tx=0x7f347c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:26.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c004b20 con 0x7f348410f3b0 2024-09-10T22:57:26.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f347c004cc0 con 0x7f348410f3b0 2024-09-10T22:57:26.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c010690 con 0x7f348410f3b0 2024-09-10T22:57:26.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34841a1760 con 0x7f348410f3b0 2024-09-10T22:57:26.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.608+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a7e20 con 0x7f348410f3b0 2024-09-10T22:57:26.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.609+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34840758c0 con 0x7f348410f3b0 2024-09-10T22:57:26.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.611+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f347c005540 con 0x7f348410f3b0 2024-09-10T22:57:26.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.613+0000 7f3480c00640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 0x7f34680b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:26.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.613+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f347c0e6840 con 0x7f348410f3b0 2024-09-10T22:57:26.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.613+0000 7f3483400640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 0x7f34680b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:26.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.614+0000 7f3483400640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 0x7f34680b7900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3474009bd0 tx=0x7f3474005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:26.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.616+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f347c0ad230 con 0x7f348410f3b0 2024-09-10T22:57:26.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.840+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3484100b50 con 0x7f34680b5440 2024-09-10T22:57:26.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.845+0000 7f3480c00640 1 -- 172.21.15.67:0/2511163059 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3484100b50 con 0x7f34680b5440 2024-09-10T22:57:26.847 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_1/cb60a4b8-6981-4bd5-a512-19427e4eaf7a 2024-09-10T22:57:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 msgr2=0x7f34680b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 0x7f34680b7900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3474009bd0 tx=0x7f3474005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 msgr2=0x7f34841a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f347c00eb40 tx=0x7f347c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 shutdown_connections 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34680b5440 0x7f34680b7900 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348410f3b0 0x7f34841a0ca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3484075e00 0x7f348419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 --2- 172.21.15.67:0/2511163059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34840754e0 0x7f348419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.847+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 >> 172.21.15.67:0/2511163059 conn(0x7f34840fe830 msgr2=0x7f34840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.848+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 shutdown_connections 2024-09-10T22:57:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:26.848+0000 7f348a200640 1 -- 172.21.15.67:0/2511163059 wait complete. 2024-09-10T22:57:27.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:57:27.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:27 smithi067 ceph-mon[30939]: pgmap v340: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-10T22:57:27.757 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:27 smithi067 ceph-mon[30939]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:27 smithi092 ceph-mon[34223]: pgmap v340: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-10T22:57:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:27 smithi092 ceph-mon[34223]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:27 smithi145 ceph-mon[34397]: pgmap v340: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-10T22:57:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:27 smithi145 ceph-mon[34397]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:57:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 -- 172.21.15.67:0/3102860617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075f40 msgr2=0x7f95a410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 --2- 172.21.15.67:0/3102860617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075f40 0x7f95a410ec60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f958c009900 tx=0x7f958c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 -- 172.21.15.67:0/3102860617 shutdown_connections 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 --2- 172.21.15.67:0/3102860617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410f230 0x7f95a4111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 --2- 172.21.15.67:0/3102860617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075f40 0x7f95a410ec60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 --2- 172.21.15.67:0/3102860617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a4075540 0x7f95a4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 -- 172.21.15.67:0/3102860617 >> 172.21.15.67:0/3102860617 conn(0x7f95a40fe6c0 msgr2=0x7f95a4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 -- 172.21.15.67:0/3102860617 shutdown_connections 2024-09-10T22:57:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.703+0000 7f95a3200640 1 -- 172.21.15.67:0/3102860617 wait complete. 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.704+0000 7f95a3200640 1 Processor -- start 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.704+0000 7f95a3200640 1 -- start start 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4075f40 0x7f95a419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 0x7f95a41a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114c90 con 0x7f95a4075540 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114b10 con 0x7f95a4075f40 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114e10 con 0x7f95a410f230 2024-09-10T22:57:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36582/0 (socket says 172.21.15.67:36582) 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 -- 172.21.15.67:0/3652725084 learned_addr learned my addr 172.21.15.67:0/3652725084 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f9599400640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4075f40 0x7f95a419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a2800640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 0x7f95a41a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 -- 172.21.15.67:0/3652725084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 msgr2=0x7f95a41a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 0x7f95a41a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 -- 172.21.15.67:0/3652725084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4075f40 msgr2=0x7f95a419b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4075f40 0x7f95a419b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.705+0000 7f95a1e00640 1 -- 172.21.15.67:0/3652725084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a1250 con 0x7f95a4075540 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f95a2800640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 0x7f95a41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f95a1e00640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f959400c920 tx=0x7f959400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594007c70 con 0x7f95a4075540 2024-09-10T22:57:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9594007e10 con 0x7f95a4075540 2024-09-10T22:57:28.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594018710 con 0x7f95a4075540 2024-09-10T22:57:28.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a1540 con 0x7f95a4075540 2024-09-10T22:57:28.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.706+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a7dd0 con 0x7f95a4075540 2024-09-10T22:57:28.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.707+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4075920 con 0x7f95a4075540 2024-09-10T22:57:28.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.710+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f959400f450 con 0x7f95a4075540 2024-09-10T22:57:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.711+0000 7f959b400640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 0x7f95880b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.711+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f95940e6490 con 0x7f95a4075540 2024-09-10T22:57:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.711+0000 7f9599400640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 0x7f95880b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.712+0000 7f9599400640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 0x7f95880b79a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f958c0057c0 tx=0x7f958c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.713+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9594014030 con 0x7f95a4075540 2024-09-10T22:57:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.951+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95a410a320 con 0x7f95a4075540 2024-09-10T22:57:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.952+0000 7f959b400640 1 -- 172.21.15.67:0/3652725084 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f95940acff0 con 0x7f95a4075540 2024-09-10T22:57:28.954 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:28.955 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:57:28.955 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 msgr2=0x7f95880b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 0x7f95880b79a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f958c0057c0 tx=0x7f958c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 msgr2=0x7f95a419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f959400c920 tx=0x7f959400cde0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 shutdown_connections 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f95880b54e0 0x7f95880b79a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95a410f230 0x7f95a41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4075f40 0x7f95a419b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 --2- 172.21.15.67:0/3652725084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95a4075540 0x7f95a419b1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.955+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 >> 172.21.15.67:0/3652725084 conn(0x7f95a40fe6c0 msgr2=0x7f95a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.956+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 shutdown_connections 2024-09-10T22:57:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:28.956+0000 7f95a3200640 1 -- 172.21.15.67:0/3652725084 wait complete. 2024-09-10T22:57:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:29 smithi092 ceph-mon[34223]: pgmap v341: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-10T22:57:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:29 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3652725084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:29.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:29 smithi067 ceph-mon[30939]: pgmap v341: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-10T22:57:29.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:29 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3652725084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:29.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:57:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:29 smithi145 ceph-mon[34397]: pgmap v341: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-10T22:57:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:29 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3652725084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:30.206 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:30.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 -- 172.21.15.67:0/3819705528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010a9b0 msgr2=0x7fe69010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 --2- 172.21.15.67:0/3819705528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010a9b0 0x7fe69010adf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe678009900 tx=0x7fe67802a410 comp rx=0 tx=0).stop 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 -- 172.21.15.67:0/3819705528 shutdown_connections 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 --2- 172.21.15.67:0/3819705528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010b330 0x7fe690111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 --2- 172.21.15.67:0/3819705528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010a9b0 0x7fe69010adf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 --2- 172.21.15.67:0/3819705528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 0x7fe690105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 -- 172.21.15.67:0/3819705528 >> 172.21.15.67:0/3819705528 conn(0x7fe690100a50 msgr2=0x7fe690102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 -- 172.21.15.67:0/3819705528 shutdown_connections 2024-09-10T22:57:30.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.898+0000 7fe694e00640 1 -- 172.21.15.67:0/3819705528 wait complete. 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 Processor -- start 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 -- start start 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 0x7fe69019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010a9b0 0x7fe69019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114e10 con 0x7fe69010b330 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114c90 con 0x7fe690104fa0 2024-09-10T22:57:30.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.900+0000 7fe694e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114f90 con 0x7fe69010a9b0 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46212/0 (socket says 172.21.15.67:46212) 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 -- 172.21.15.67:0/1949428851 learned_addr learned my addr 172.21.15.67:0/1949428851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68de00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 0x7fe69019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe687e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010a9b0 0x7fe69019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 -- 172.21.15.67:0/1949428851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010a9b0 msgr2=0x7fe69019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010a9b0 0x7fe69019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 -- 172.21.15.67:0/1949428851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 msgr2=0x7fe69019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 0x7fe69019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.901+0000 7fe68e800640 1 -- 172.21.15.67:0/1949428851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6901a1410 con 0x7fe69010b330 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe68e800640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68800eb10 tx=0x7fe68800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004b20 con 0x7fe69010b330 2024-09-10T22:57:30.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe688004cc0 con 0x7fe69010b330 2024-09-10T22:57:30.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6901a1700 con 0x7fe69010b330 2024-09-10T22:57:30.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688010690 con 0x7fe69010b330 2024-09-10T22:57:30.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.902+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a7dc0 con 0x7fe69010b330 2024-09-10T22:57:30.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.903+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690105380 con 0x7fe69010b330 2024-09-10T22:57:30.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.905+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fe688005020 con 0x7fe69010b330 2024-09-10T22:57:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.907+0000 7fe686a00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 0x7fe6740b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.907+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fe6880e66f0 con 0x7fe69010b330 2024-09-10T22:57:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.907+0000 7fe68de00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 0x7fe6740b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.908+0000 7fe68de00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 0x7fe6740b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe680009bd0 tx=0x7fe680005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:30.910+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6880ad140 con 0x7fe69010b330 2024-09-10T22:57:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.112+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69002e5c0 con 0x7fe69010b330 2024-09-10T22:57:31.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.113+0000 7fe686a00640 1 -- 172.21.15.67:0/1949428851 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24488 (secure 0 0 0) 0x7fe6880aca40 con 0x7fe69010b330 2024-09-10T22:57:31.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:31.116 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:56:30.957387+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 msgr2=0x7fe6740b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 0x7fe6740b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe680009bd0 tx=0x7fe680005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 msgr2=0x7fe6901a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68800eb10 tx=0x7fe68800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 shutdown_connections 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6740b53c0 0x7fe6740b7880 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe69010b330 0x7fe6901a0c40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69010a9b0 0x7fe69019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 --2- 172.21.15.67:0/1949428851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe690104fa0 0x7fe69019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 >> 172.21.15.67:0/1949428851 conn(0x7fe690100a50 msgr2=0x7fe690102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 shutdown_connections 2024-09-10T22:57:31.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:31.116+0000 7fe694e00640 1 -- 172.21.15.67:0/1949428851 wait complete. 2024-09-10T22:57:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:31 smithi092 ceph-mon[34223]: pgmap v342: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T22:57:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:31 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1949428851' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:31 smithi067 ceph-mon[30939]: pgmap v342: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T22:57:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:31 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1949428851' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:31 smithi145 ceph-mon[34397]: pgmap v342: 129 pgs: 129 active+clean; 623 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T22:57:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:31 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1949428851' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:32.067 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:57:32.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:57:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:32.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-10T22:57:32.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:32 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-10T22:57:32.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:32 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:57:32.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]} : dispatch 2024-09-10T22:57:32.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-10T22:57:32.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:32 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:57:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 -- 172.21.15.67:0/4280556104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c88104fa0 msgr2=0x7f6c88105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 --2- 172.21.15.67:0/4280556104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c88104fa0 0x7f6c88105380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009900 tx=0x7f6c7802a410 comp rx=0 tx=0).stop 2024-09-10T22:57:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 -- 172.21.15.67:0/4280556104 shutdown_connections 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 --2- 172.21.15.67:0/4280556104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 --2- 172.21.15.67:0/4280556104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c8810a9b0 0x7f6c8810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 --2- 172.21.15.67:0/4280556104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c88104fa0 0x7f6c88105380 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 -- 172.21.15.67:0/4280556104 >> 172.21.15.67:0/4280556104 conn(0x7f6c88100a50 msgr2=0x7f6c88102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 -- 172.21.15.67:0/4280556104 shutdown_connections 2024-09-10T22:57:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.058+0000 7f6c8d800640 1 -- 172.21.15.67:0/4280556104 wait complete. 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.059+0000 7f6c8d800640 1 Processor -- start 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.059+0000 7f6c8d800640 1 -- start start 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c88104fa0 0x7f6c880776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88114df0 con 0x7f6c8810a9b0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88114c70 con 0x7f6c8810b330 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c8d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88114f70 con 0x7f6c88104fa0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46234/0 (socket says 172.21.15.67:46234) 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 -- 172.21.15.67:0/3967688444 learned_addr learned my addr 172.21.15.67:0/3967688444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c86a00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c88104fa0 0x7f6c880776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c87400640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 -- 172.21.15.67:0/3967688444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c88104fa0 msgr2=0x7f6c880776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c88104fa0 0x7f6c880776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 -- 172.21.15.67:0/3967688444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 msgr2=0x7f6c88078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88078130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c7fe00640 1 -- 172.21.15.67:0/3967688444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c881aba30 con 0x7f6c8810a9b0 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.060+0000 7f6c87400640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c7fe00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000d950 tx=0x7f6c7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70017030 con 0x7f6c8810a9b0 2024-09-10T22:57:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c70004020 con 0x7f6c8810a9b0 2024-09-10T22:57:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700055d0 con 0x7f6c8810a9b0 2024-09-10T22:57:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c881abd20 con 0x7f6c8810a9b0 2024-09-10T22:57:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.061+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881ac1b0 con 0x7f6c8810a9b0 2024-09-10T22:57:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.062+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c88105380 con 0x7f6c8810a9b0 2024-09-10T22:57:33.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.065+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6c70009a30 con 0x7f6c8810a9b0 2024-09-10T22:57:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.066+0000 7f6c7f400640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 0x7f6c6c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.066+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9286+0+0 (secure 0 0 0) 0x7f6c700e5c40 con 0x7f6c8810a9b0 2024-09-10T22:57:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.066+0000 7f6c86a00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 0x7f6c6c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.067+0000 7f6c86a00640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 0x7f6c6c0b7900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.070+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6c700ac520 con 0x7f6c8810a9b0 2024-09-10T22:57:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.261+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c8802e6f0 con 0x7f6c6c0b5440 2024-09-10T22:57:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.268+0000 7f6c7f400640 1 -- 172.21.15.67:0/3967688444 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340602 (secure 0 0 0) 0x7f6c8802e6f0 con 0x7f6c6c0b5440 2024-09-10T22:57:33.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:33.276 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:57:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 msgr2=0x7f6c6c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 0x7f6c6c0b7900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78005a20 comp rx=0 tx=0).stop 2024-09-10T22:57:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 msgr2=0x7f6c88077bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000d950 tx=0x7f6c7000de10 comp rx=0 tx=0).stop 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 shutdown_connections 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6c6c0b5440 0x7f6c6c0b7900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8810b330 0x7f6c88078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8810a9b0 0x7f6c88077bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 --2- 172.21.15.67:0/3967688444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c88104fa0 0x7f6c880776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.272+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 >> 172.21.15.67:0/3967688444 conn(0x7f6c88100a50 msgr2=0x7f6c88101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.273+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 shutdown_connections 2024-09-10T22:57:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:33.273+0000 7f6c8d800640 1 -- 172.21.15.67:0/3967688444 wait complete. 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: pgmap v343: 129 pgs: 129 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:57:33.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:33 smithi092 ceph-mon[34223]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:57:33.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: pgmap v343: 129 pgs: 129 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-10T22:57:33.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-09-10T22:57:33.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-10T22:57:33.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:57:33.795 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:33 smithi067 ceph-mon[30939]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:57:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: pgmap v343: 129 pgs: 129 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-10T22:57:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-09-10T22:57:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:57:33.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:33 smithi145 ceph-mon[34397]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:57:34.300 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T22:57:32.474750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647007,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":216,"num_read_kb":198,"num_write":287,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356448,"kb_used_data":11360,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585376,"statfs":{"total":1151940427776,"available":1151575425024,"internally_reserved":0,"allocated":11632640,"data_stored":7672847,"data_compressed":319995,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28062,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":72,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.402750"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425984+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:10.425984+0000","last_peered":"2024-09-10T22:57:10.425984+0000","last_clean":"2024-09-10T22:57:10.425984+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:10.425984+0000","last_undegraded":"2024-09-10T22:57:10.425984+0000","last_fullsized":"2024-09-10T22:57:10.425984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268493+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:56:31.268493+0000","last_peered":"2024-09-10T22:56:31.268493+0000","last_clean":"2024-09-10T22:56:31.268493+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:56:31.268493+0000","last_undegraded":"2024-09-10T22:56:31.268493+0000","last_fullsized":"2024-09-10T22:56:31.268493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267878+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:56:31.267878+0000","last_peered":"2024-09-10T22:56:31.267878+0000","last_clean":"2024-09-10T22:56:31.267878+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:56:31.267878+0000","last_undegraded":"2024-09-10T22:56:31.267878+0000","last_fullsized":"2024-09-10T22:56:31.267878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269504+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:56:31.269504+0000","last_peered":"2024-09-10T22:56:31.269504+0000","last_clean":"2024-09-10T22:56:31.269504+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:56:31.269504+0000","last_undegraded":"2024-09-10T22:56:31.269504+0000","last_fullsized":"2024-09-10T22:56:31.269504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270294+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:56:31.270294+0000","last_peered":"2024-09-10T22:56:31.270294+0000","last_clean":"2024-09-10T22:56:31.270294+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:56:31.270294+0000","last_undegraded":"2024-09-10T22:56:31.270294+0000","last_fullsized":"2024-09-10T22:56:31.270294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114275+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:56:31.114275+0000","last_peered":"2024-09-10T22:56:31.114275+0000","last_clean":"2024-09-10T22:56:31.114275+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:56:31.114275+0000","last_undegraded":"2024-09-10T22:56:31.114275+0000","last_fullsized":"2024-09-10T22:56:31.114275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.423890+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:10.423890+0000","last_peered":"2024-09-10T22:57:10.423890+0000","last_clean":"2024-09-10T22:57:10.423890+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:10.423890+0000","last_undegraded":"2024-09-10T22:57:10.423890+0000","last_fullsized":"2024-09-10T22:57:10.423890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974481+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:56:30.974481+0000","last_peered":"2024-09-10T22:56:30.974481+0000","last_clean":"2024-09-10T22:56:30.974481+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:56:30.974481+0000","last_undegraded":"2024-09-10T22:56:30.974481+0000","last_fullsized":"2024-09-10T22:56:30.974481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114523+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:56:31.114523+0000","last_peered":"2024-09-10T22:56:31.114523+0000","last_clean":"2024-09-10T22:56:31.114523+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:56:31.114523+0000","last_undegraded":"2024-09-10T22:56:31.114523+0000","last_fullsized":"2024-09-10T22:56:31.114523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268383+0000","last_change":"2024-09-10T22:56:16.891206+0000","last_active":"2024-09-10T22:56:31.268383+0000","last_peered":"2024-09-10T22:56:31.268383+0000","last_clean":"2024-09-10T22:56:31.268383+0000","last_became_active":"2024-09-10T22:56:16.890866+0000","last_became_peered":"2024-09-10T22:56:16.890866+0000","last_unstale":"2024-09-10T22:56:31.268383+0000","last_undegraded":"2024-09-10T22:56:31.268383+0000","last_fullsized":"2024-09-10T22:56:31.268383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:54.805629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.975151+0000","last_change":"2024-09-10T22:56:18.923375+0000","last_active":"2024-09-10T22:56:30.975151+0000","last_peered":"2024-09-10T22:56:30.975151+0000","last_clean":"2024-09-10T22:56:30.975151+0000","last_became_active":"2024-09-10T22:56:18.921985+0000","last_became_peered":"2024-09-10T22:56:18.921985+0000","last_unstale":"2024-09-10T22:56:30.975151+0000","last_undegraded":"2024-09-10T22:56:30.975151+0000","last_fullsized":"2024-09-10T22:56:30.975151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:14.061052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114411+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:56:31.114411+0000","last_peered":"2024-09-10T22:56:31.114411+0000","last_clean":"2024-09-10T22:56:31.114411+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:56:31.114411+0000","last_undegraded":"2024-09-10T22:56:31.114411+0000","last_fullsized":"2024-09-10T22:56:31.114411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268569+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:56:31.268569+0000","last_peered":"2024-09-10T22:56:31.268569+0000","last_clean":"2024-09-10T22:56:31.268569+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:56:31.268569+0000","last_undegraded":"2024-09-10T22:56:31.268569+0000","last_fullsized":"2024-09-10T22:56:31.268569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.416212+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:09.416212+0000","last_peered":"2024-09-10T22:57:09.416212+0000","last_clean":"2024-09-10T22:57:09.416212+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:09.416212+0000","last_undegraded":"2024-09-10T22:57:09.416212+0000","last_fullsized":"2024-09-10T22:57:09.416212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114671+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:56:31.114671+0000","last_peered":"2024-09-10T22:56:31.114671+0000","last_clean":"2024-09-10T22:56:31.114671+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:56:31.114671+0000","last_undegraded":"2024-09-10T22:56:31.114671+0000","last_fullsized":"2024-09-10T22:56:31.114671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269152+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:56:31.269152+0000","last_peered":"2024-09-10T22:56:31.269152+0000","last_clean":"2024-09-10T22:56:31.269152+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:56:31.269152+0000","last_undegraded":"2024-09-10T22:56:31.269152+0000","last_fullsized":"2024-09-10T22:56:31.269152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269001+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:56:31.269001+0000","last_peered":"2024-09-10T22:56:31.269001+0000","last_clean":"2024-09-10T22:56:31.269001+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:56:31.269001+0000","last_undegraded":"2024-09-10T22:56:31.269001+0000","last_fullsized":"2024-09-10T22:56:31.269001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114845+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:56:31.114845+0000","last_peered":"2024-09-10T22:56:31.114845+0000","last_clean":"2024-09-10T22:56:31.114845+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:56:31.114845+0000","last_undegraded":"2024-09-10T22:56:31.114845+0000","last_fullsized":"2024-09-10T22:56:31.114845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215317+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:56:31.215317+0000","last_peered":"2024-09-10T22:56:31.215317+0000","last_clean":"2024-09-10T22:56:31.215317+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:56:31.215317+0000","last_undegraded":"2024-09-10T22:56:31.215317+0000","last_fullsized":"2024-09-10T22:56:31.215317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.414348+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:09.414348+0000","last_peered":"2024-09-10T22:57:09.414348+0000","last_clean":"2024-09-10T22:57:09.414348+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:09.414348+0000","last_undegraded":"2024-09-10T22:57:09.414348+0000","last_fullsized":"2024-09-10T22:57:09.414348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269937+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:56:31.269937+0000","last_peered":"2024-09-10T22:56:31.269937+0000","last_clean":"2024-09-10T22:56:31.269937+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:56:31.269937+0000","last_undegraded":"2024-09-10T22:56:31.269937+0000","last_fullsized":"2024-09-10T22:56:31.269937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267439+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:56:31.267439+0000","last_peered":"2024-09-10T22:56:31.267439+0000","last_clean":"2024-09-10T22:56:31.267439+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:56:31.267439+0000","last_undegraded":"2024-09-10T22:56:31.267439+0000","last_fullsized":"2024-09-10T22:56:31.267439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.415183+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:09.415183+0000","last_peered":"2024-09-10T22:57:09.415183+0000","last_clean":"2024-09-10T22:57:09.415183+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:09.415183+0000","last_undegraded":"2024-09-10T22:57:09.415183+0000","last_fullsized":"2024-09-10T22:57:09.415183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114846+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:56:31.114846+0000","last_peered":"2024-09-10T22:56:31.114846+0000","last_clean":"2024-09-10T22:56:31.114846+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:56:31.114846+0000","last_undegraded":"2024-09-10T22:56:31.114846+0000","last_fullsized":"2024-09-10T22:56:31.114846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270007+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:56:31.270007+0000","last_peered":"2024-09-10T22:56:31.270007+0000","last_clean":"2024-09-10T22:56:31.270007+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:56:31.270007+0000","last_undegraded":"2024-09-10T22:56:31.270007+0000","last_fullsized":"2024-09-10T22:56:31.270007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.414042+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:09.414042+0000","last_peered":"2024-09-10T22:57:09.414042+0000","last_clean":"2024-09-10T22:57:09.414042+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:09.414042+0000","last_undegraded":"2024-09-10T22:57:09.414042+0000","last_fullsized":"2024-09-10T22:57:09.414042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268138+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:56:31.268138+0000","last_peered":"2024-09-10T22:56:31.268138+0000","last_clean":"2024-09-10T22:56:31.268138+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:56:31.268138+0000","last_undegraded":"2024-09-10T22:56:31.268138+0000","last_fullsized":"2024-09-10T22:56:31.268138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269611+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:56:31.269611+0000","last_peered":"2024-09-10T22:56:31.269611+0000","last_clean":"2024-09-10T22:56:31.269611+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:56:31.269611+0000","last_undegraded":"2024-09-10T22:56:31.269611+0000","last_fullsized":"2024-09-10T22:56:31.269611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.916037+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:27.916037+0000","last_peered":"2024-09-10T22:57:27.916037+0000","last_clean":"2024-09-10T22:57:27.916037+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:27.916037+0000","last_undegraded":"2024-09-10T22:57:27.916037+0000","last_fullsized":"2024-09-10T22:57:27.916037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268582+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:56:31.268582+0000","last_peered":"2024-09-10T22:56:31.268582+0000","last_clean":"2024-09-10T22:56:31.268582+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:56:31.268582+0000","last_undegraded":"2024-09-10T22:56:31.268582+0000","last_fullsized":"2024-09-10T22:56:31.268582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973506+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:56:30.973506+0000","last_peered":"2024-09-10T22:56:30.973506+0000","last_clean":"2024-09-10T22:56:30.973506+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:56:30.973506+0000","last_undegraded":"2024-09-10T22:56:30.973506+0000","last_fullsized":"2024-09-10T22:56:30.973506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.413823+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:09.413823+0000","last_peered":"2024-09-10T22:57:09.413823+0000","last_clean":"2024-09-10T22:57:09.413823+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:09.413823+0000","last_undegraded":"2024-09-10T22:57:09.413823+0000","last_fullsized":"2024-09-10T22:57:09.413823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215625+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:56:31.215625+0000","last_peered":"2024-09-10T22:56:31.215625+0000","last_clean":"2024-09-10T22:56:31.215625+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:56:31.215625+0000","last_undegraded":"2024-09-10T22:56:31.215625+0000","last_fullsized":"2024-09-10T22:56:31.215625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268840+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:56:31.268840+0000","last_peered":"2024-09-10T22:56:31.268840+0000","last_clean":"2024-09-10T22:56:31.268840+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:56:31.268840+0000","last_undegraded":"2024-09-10T22:56:31.268840+0000","last_fullsized":"2024-09-10T22:56:31.268840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269684+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:56:31.269684+0000","last_peered":"2024-09-10T22:56:31.269684+0000","last_clean":"2024-09-10T22:56:31.269684+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:56:31.269684+0000","last_undegraded":"2024-09-10T22:56:31.269684+0000","last_fullsized":"2024-09-10T22:56:31.269684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267571+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:56:31.267571+0000","last_peered":"2024-09-10T22:56:31.267571+0000","last_clean":"2024-09-10T22:56:31.267571+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:56:31.267571+0000","last_undegraded":"2024-09-10T22:56:31.267571+0000","last_fullsized":"2024-09-10T22:56:31.267571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268978+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:56:31.268978+0000","last_peered":"2024-09-10T22:56:31.268978+0000","last_clean":"2024-09-10T22:56:31.268978+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:56:31.268978+0000","last_undegraded":"2024-09-10T22:56:31.268978+0000","last_fullsized":"2024-09-10T22:56:31.268978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268776+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:56:31.268776+0000","last_peered":"2024-09-10T22:56:31.268776+0000","last_clean":"2024-09-10T22:56:31.268776+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:56:31.268776+0000","last_undegraded":"2024-09-10T22:56:31.268776+0000","last_fullsized":"2024-09-10T22:56:31.268776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973373+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:56:30.973373+0000","last_peered":"2024-09-10T22:56:30.973373+0000","last_clean":"2024-09-10T22:56:30.973373+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:56:30.973373+0000","last_undegraded":"2024-09-10T22:56:30.973373+0000","last_fullsized":"2024-09-10T22:56:30.973373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974431+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:56:30.974431+0000","last_peered":"2024-09-10T22:56:30.974431+0000","last_clean":"2024-09-10T22:56:30.974431+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:56:30.974431+0000","last_undegraded":"2024-09-10T22:56:30.974431+0000","last_fullsized":"2024-09-10T22:56:30.974431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:28.466035+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:57:28.466035+0000","last_peered":"2024-09-10T22:57:28.466035+0000","last_clean":"2024-09-10T22:57:28.466035+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:57:28.466035+0000","last_undegraded":"2024-09-10T22:57:28.466035+0000","last_fullsized":"2024-09-10T22:57:28.466035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973586+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:56:30.973586+0000","last_peered":"2024-09-10T22:56:30.973586+0000","last_clean":"2024-09-10T22:56:30.973586+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:56:30.973586+0000","last_undegraded":"2024-09-10T22:56:30.973586+0000","last_fullsized":"2024-09-10T22:56:30.973586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.972886+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:56:30.972886+0000","last_peered":"2024-09-10T22:56:30.972886+0000","last_clean":"2024-09-10T22:56:30.972886+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:56:30.972886+0000","last_undegraded":"2024-09-10T22:56:30.972886+0000","last_fullsized":"2024-09-10T22:56:30.972886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425423+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:10.425423+0000","last_peered":"2024-09-10T22:57:10.425423+0000","last_clean":"2024-09-10T22:57:10.425423+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:10.425423+0000","last_undegraded":"2024-09-10T22:57:10.425423+0000","last_fullsized":"2024-09-10T22:57:10.425423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.975247+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:56:30.975247+0000","last_peered":"2024-09-10T22:56:30.975247+0000","last_clean":"2024-09-10T22:56:30.975247+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:56:30.975247+0000","last_undegraded":"2024-09-10T22:56:30.975247+0000","last_fullsized":"2024-09-10T22:56:30.975247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269073+0000","last_change":"2024-09-10T22:56:18.920310+0000","last_active":"2024-09-10T22:56:31.269073+0000","last_peered":"2024-09-10T22:56:31.269073+0000","last_clean":"2024-09-10T22:56:31.269073+0000","last_became_active":"2024-09-10T22:56:18.918860+0000","last_became_peered":"2024-09-10T22:56:18.918860+0000","last_unstale":"2024-09-10T22:56:31.269073+0000","last_undegraded":"2024-09-10T22:56:31.269073+0000","last_fullsized":"2024-09-10T22:56:31.269073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:33.506508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:12.434910+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:12.434910+0000","last_peered":"2024-09-10T22:57:12.434910+0000","last_clean":"2024-09-10T22:57:12.434910+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:12.434910+0000","last_undegraded":"2024-09-10T22:57:12.434910+0000","last_fullsized":"2024-09-10T22:57:12.434910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":109,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268394+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:56:31.268394+0000","last_peered":"2024-09-10T22:56:31.268394+0000","last_clean":"2024-09-10T22:56:31.268394+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:56:31.268394+0000","last_undegraded":"2024-09-10T22:56:31.268394+0000","last_fullsized":"2024-09-10T22:56:31.268394+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269113+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:56:31.269113+0000","last_peered":"2024-09-10T22:56:31.269113+0000","last_clean":"2024-09-10T22:56:31.269113+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:56:31.269113+0000","last_undegraded":"2024-09-10T22:56:31.269113+0000","last_fullsized":"2024-09-10T22:56:31.269113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269283+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:56:31.269283+0000","last_peered":"2024-09-10T22:56:31.269283+0000","last_clean":"2024-09-10T22:56:31.269283+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:56:31.269283+0000","last_undegraded":"2024-09-10T22:56:31.269283+0000","last_fullsized":"2024-09-10T22:56:31.269283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425898+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:10.425898+0000","last_peered":"2024-09-10T22:57:10.425898+0000","last_clean":"2024-09-10T22:57:10.425898+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:10.425898+0000","last_undegraded":"2024-09-10T22:57:10.425898+0000","last_fullsized":"2024-09-10T22:57:10.425898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974361+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:56:30.974361+0000","last_peered":"2024-09-10T22:56:30.974361+0000","last_clean":"2024-09-10T22:56:30.974361+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:56:30.974361+0000","last_undegraded":"2024-09-10T22:56:30.974361+0000","last_fullsized":"2024-09-10T22:56:30.974361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268717+0000","last_change":"2024-09-10T22:56:18.913029+0000","last_active":"2024-09-10T22:56:31.268717+0000","last_peered":"2024-09-10T22:56:31.268717+0000","last_clean":"2024-09-10T22:56:31.268717+0000","last_became_active":"2024-09-10T22:56:18.912587+0000","last_became_peered":"2024-09-10T22:56:18.912587+0000","last_unstale":"2024-09-10T22:56:31.268717+0000","last_undegraded":"2024-09-10T22:56:31.268717+0000","last_fullsized":"2024-09-10T22:56:31.268717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:02.256239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.975500+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:56:30.975500+0000","last_peered":"2024-09-10T22:56:30.975500+0000","last_clean":"2024-09-10T22:56:30.975500+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:56:30.975500+0000","last_undegraded":"2024-09-10T22:56:30.975500+0000","last_fullsized":"2024-09-10T22:56:30.975500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269468+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:56:31.269468+0000","last_peered":"2024-09-10T22:56:31.269468+0000","last_clean":"2024-09-10T22:56:31.269468+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:56:31.269468+0000","last_undegraded":"2024-09-10T22:56:31.269468+0000","last_fullsized":"2024-09-10T22:56:31.269468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268930+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:56:31.268930+0000","last_peered":"2024-09-10T22:56:31.268930+0000","last_clean":"2024-09-10T22:56:31.268930+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:56:31.268930+0000","last_undegraded":"2024-09-10T22:56:31.268930+0000","last_fullsized":"2024-09-10T22:56:31.268930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973510+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:56:30.973510+0000","last_peered":"2024-09-10T22:56:30.973510+0000","last_clean":"2024-09-10T22:56:30.973510+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:56:30.973510+0000","last_undegraded":"2024-09-10T22:56:30.973510+0000","last_fullsized":"2024-09-10T22:56:30.973510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974212+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:56:30.974212+0000","last_peered":"2024-09-10T22:56:30.974212+0000","last_clean":"2024-09-10T22:56:30.974212+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:56:30.974212+0000","last_undegraded":"2024-09-10T22:56:30.974212+0000","last_fullsized":"2024-09-10T22:56:30.974212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215700+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:56:31.215700+0000","last_peered":"2024-09-10T22:56:31.215700+0000","last_clean":"2024-09-10T22:56:31.215700+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:56:31.215700+0000","last_undegraded":"2024-09-10T22:56:31.215700+0000","last_fullsized":"2024-09-10T22:56:31.215700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:11.418870+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:11.418870+0000","last_peered":"2024-09-10T22:57:11.418870+0000","last_clean":"2024-09-10T22:57:11.418870+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:11.418870+0000","last_undegraded":"2024-09-10T22:57:11.418870+0000","last_fullsized":"2024-09-10T22:57:11.418870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974511+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:56:30.974511+0000","last_peered":"2024-09-10T22:56:30.974511+0000","last_clean":"2024-09-10T22:56:30.974511+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:56:30.974511+0000","last_undegraded":"2024-09-10T22:56:30.974511+0000","last_fullsized":"2024-09-10T22:56:30.974511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974517+0000","last_change":"2024-09-10T22:56:18.927689+0000","last_active":"2024-09-10T22:56:30.974517+0000","last_peered":"2024-09-10T22:56:30.974517+0000","last_clean":"2024-09-10T22:56:30.974517+0000","last_became_active":"2024-09-10T22:56:18.926585+0000","last_became_peered":"2024-09-10T22:56:18.926585+0000","last_unstale":"2024-09-10T22:56:30.974517+0000","last_undegraded":"2024-09-10T22:56:30.974517+0000","last_fullsized":"2024-09-10T22:56:30.974517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:57.974169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425826+0000","last_change":"2024-09-10T22:56:17.606383+0000","last_active":"2024-09-10T22:57:10.425826+0000","last_peered":"2024-09-10T22:57:10.425826+0000","last_clean":"2024-09-10T22:57:10.425826+0000","last_became_active":"2024-09-10T22:56:17.605996+0000","last_became_peered":"2024-09-10T22:56:17.605996+0000","last_unstale":"2024-09-10T22:57:10.425826+0000","last_undegraded":"2024-09-10T22:57:10.425826+0000","last_fullsized":"2024-09-10T22:57:10.425826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:00.991079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973575+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:56:30.973575+0000","last_peered":"2024-09-10T22:56:30.973575+0000","last_clean":"2024-09-10T22:56:30.973575+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:56:30.973575+0000","last_undegraded":"2024-09-10T22:56:30.973575+0000","last_fullsized":"2024-09-10T22:56:30.973575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268102+0000","last_change":"2024-09-10T22:56:18.923145+0000","last_active":"2024-09-10T22:56:31.268102+0000","last_peered":"2024-09-10T22:56:31.268102+0000","last_clean":"2024-09-10T22:56:31.268102+0000","last_became_active":"2024-09-10T22:56:18.918596+0000","last_became_peered":"2024-09-10T22:56:18.918596+0000","last_unstale":"2024-09-10T22:56:31.268102+0000","last_undegraded":"2024-09-10T22:56:31.268102+0000","last_fullsized":"2024-09-10T22:56:31.268102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:31.472980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.464935+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:57:27.464935+0000","last_peered":"2024-09-10T22:57:27.464935+0000","last_clean":"2024-09-10T22:57:27.464935+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:57:27.464935+0000","last_undegraded":"2024-09-10T22:57:27.464935+0000","last_fullsized":"2024-09-10T22:57:27.464935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270217+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:56:31.270217+0000","last_peered":"2024-09-10T22:56:31.270217+0000","last_clean":"2024-09-10T22:56:31.270217+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:56:31.270217+0000","last_undegraded":"2024-09-10T22:56:31.270217+0000","last_fullsized":"2024-09-10T22:56:31.270217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268583+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:56:31.268583+0000","last_peered":"2024-09-10T22:56:31.268583+0000","last_clean":"2024-09-10T22:56:31.268583+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:56:31.268583+0000","last_undegraded":"2024-09-10T22:56:31.268583+0000","last_fullsized":"2024-09-10T22:56:31.268583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.412876+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:09.412876+0000","last_peered":"2024-09-10T22:57:09.412876+0000","last_clean":"2024-09-10T22:57:09.412876+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:09.412876+0000","last_undegraded":"2024-09-10T22:57:09.412876+0000","last_fullsized":"2024-09-10T22:57:09.412876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215461+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:56:31.215461+0000","last_peered":"2024-09-10T22:56:31.215461+0000","last_clean":"2024-09-10T22:56:31.215461+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:56:31.215461+0000","last_undegraded":"2024-09-10T22:56:31.215461+0000","last_fullsized":"2024-09-10T22:56:31.215461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973005+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:56:30.973005+0000","last_peered":"2024-09-10T22:56:30.973005+0000","last_clean":"2024-09-10T22:56:30.973005+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:56:30.973005+0000","last_undegraded":"2024-09-10T22:56:30.973005+0000","last_fullsized":"2024-09-10T22:56:30.973005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.424567+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:10.424567+0000","last_peered":"2024-09-10T22:57:10.424567+0000","last_clean":"2024-09-10T22:57:10.424567+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:10.424567+0000","last_undegraded":"2024-09-10T22:57:10.424567+0000","last_fullsized":"2024-09-10T22:57:10.424567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.975189+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:56:30.975189+0000","last_peered":"2024-09-10T22:56:30.975189+0000","last_clean":"2024-09-10T22:56:30.975189+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:56:30.975189+0000","last_undegraded":"2024-09-10T22:56:30.975189+0000","last_fullsized":"2024-09-10T22:56:30.975189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268147+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:56:31.268147+0000","last_peered":"2024-09-10T22:56:31.268147+0000","last_clean":"2024-09-10T22:56:31.268147+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:56:31.268147+0000","last_undegraded":"2024-09-10T22:56:31.268147+0000","last_fullsized":"2024-09-10T22:56:31.268147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:30.477235+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:57:30.477235+0000","last_peered":"2024-09-10T22:57:30.477235+0000","last_clean":"2024-09-10T22:57:30.477235+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:57:30.477235+0000","last_undegraded":"2024-09-10T22:57:30.477235+0000","last_fullsized":"2024-09-10T22:57:30.477235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52404,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269170+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:56:31.269170+0000","last_peered":"2024-09-10T22:56:31.269170+0000","last_clean":"2024-09-10T22:56:31.269170+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:56:31.269170+0000","last_undegraded":"2024-09-10T22:56:31.269170+0000","last_fullsized":"2024-09-10T22:56:31.269170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974638+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:56:30.974638+0000","last_peered":"2024-09-10T22:56:30.974638+0000","last_clean":"2024-09-10T22:56:30.974638+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:56:30.974638+0000","last_undegraded":"2024-09-10T22:56:30.974638+0000","last_fullsized":"2024-09-10T22:56:30.974638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.414438+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:09.414438+0000","last_peered":"2024-09-10T22:57:09.414438+0000","last_clean":"2024-09-10T22:57:09.414438+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:09.414438+0000","last_undegraded":"2024-09-10T22:57:09.414438+0000","last_fullsized":"2024-09-10T22:57:09.414438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114831+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:56:31.114831+0000","last_peered":"2024-09-10T22:56:31.114831+0000","last_clean":"2024-09-10T22:56:31.114831+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:56:31.114831+0000","last_undegraded":"2024-09-10T22:56:31.114831+0000","last_fullsized":"2024-09-10T22:56:31.114831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270319+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:56:31.270319+0000","last_peered":"2024-09-10T22:56:31.270319+0000","last_clean":"2024-09-10T22:56:31.270319+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:56:31.270319+0000","last_undegraded":"2024-09-10T22:56:31.270319+0000","last_fullsized":"2024-09-10T22:56:31.270319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.416075+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:09.416075+0000","last_peered":"2024-09-10T22:57:09.416075+0000","last_clean":"2024-09-10T22:57:09.416075+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:09.416075+0000","last_undegraded":"2024-09-10T22:57:09.416075+0000","last_fullsized":"2024-09-10T22:57:09.416075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268692+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:56:31.268692+0000","last_peered":"2024-09-10T22:56:31.268692+0000","last_clean":"2024-09-10T22:56:31.268692+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:56:31.268692+0000","last_undegraded":"2024-09-10T22:56:31.268692+0000","last_fullsized":"2024-09-10T22:56:31.268692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974698+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:56:30.974698+0000","last_peered":"2024-09-10T22:56:30.974698+0000","last_clean":"2024-09-10T22:56:30.974698+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:56:30.974698+0000","last_undegraded":"2024-09-10T22:56:30.974698+0000","last_fullsized":"2024-09-10T22:56:30.974698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:29.467786+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:57:29.467786+0000","last_peered":"2024-09-10T22:57:29.467786+0000","last_clean":"2024-09-10T22:57:29.467786+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:57:29.467786+0000","last_undegraded":"2024-09-10T22:57:29.467786+0000","last_fullsized":"2024-09-10T22:57:29.467786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268741+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:56:31.268741+0000","last_peered":"2024-09-10T22:56:31.268741+0000","last_clean":"2024-09-10T22:56:31.268741+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:56:31.268741+0000","last_undegraded":"2024-09-10T22:56:31.268741+0000","last_fullsized":"2024-09-10T22:56:31.268741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269754+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:56:31.269754+0000","last_peered":"2024-09-10T22:56:31.269754+0000","last_clean":"2024-09-10T22:56:31.269754+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:56:31.269754+0000","last_undegraded":"2024-09-10T22:56:31.269754+0000","last_fullsized":"2024-09-10T22:56:31.269754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268563+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:56:31.268563+0000","last_peered":"2024-09-10T22:56:31.268563+0000","last_clean":"2024-09-10T22:56:31.268563+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:56:31.268563+0000","last_undegraded":"2024-09-10T22:56:31.268563+0000","last_fullsized":"2024-09-10T22:56:31.268563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974714+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:56:30.974714+0000","last_peered":"2024-09-10T22:56:30.974714+0000","last_clean":"2024-09-10T22:56:30.974714+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:56:30.974714+0000","last_undegraded":"2024-09-10T22:56:30.974714+0000","last_fullsized":"2024-09-10T22:56:30.974714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215752+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:56:31.215752+0000","last_peered":"2024-09-10T22:56:31.215752+0000","last_clean":"2024-09-10T22:56:31.215752+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:56:31.215752+0000","last_undegraded":"2024-09-10T22:56:31.215752+0000","last_fullsized":"2024-09-10T22:56:31.215752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268904+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:56:31.268904+0000","last_peered":"2024-09-10T22:56:31.268904+0000","last_clean":"2024-09-10T22:56:31.268904+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:56:31.268904+0000","last_undegraded":"2024-09-10T22:56:31.268904+0000","last_fullsized":"2024-09-10T22:56:31.268904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425449+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:10.425449+0000","last_peered":"2024-09-10T22:57:10.425449+0000","last_clean":"2024-09-10T22:57:10.425449+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:10.425449+0000","last_undegraded":"2024-09-10T22:57:10.425449+0000","last_fullsized":"2024-09-10T22:57:10.425449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.426510+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:10.426510+0000","last_peered":"2024-09-10T22:57:10.426510+0000","last_clean":"2024-09-10T22:57:10.426510+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:10.426510+0000","last_undegraded":"2024-09-10T22:57:10.426510+0000","last_fullsized":"2024-09-10T22:57:10.426510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974812+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:56:30.974812+0000","last_peered":"2024-09-10T22:56:30.974812+0000","last_clean":"2024-09-10T22:56:30.974812+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:56:30.974812+0000","last_undegraded":"2024-09-10T22:56:30.974812+0000","last_fullsized":"2024-09-10T22:56:30.974812+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268800+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:56:31.268800+0000","last_peered":"2024-09-10T22:56:31.268800+0000","last_clean":"2024-09-10T22:56:31.268800+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:56:31.268800+0000","last_undegraded":"2024-09-10T22:56:31.268800+0000","last_fullsized":"2024-09-10T22:56:31.268800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.464902+0000","last_change":"2024-09-10T22:56:16.899702+0000","last_active":"2024-09-10T22:57:27.464902+0000","last_peered":"2024-09-10T22:57:27.464902+0000","last_clean":"2024-09-10T22:57:27.464902+0000","last_became_active":"2024-09-10T22:56:16.898240+0000","last_became_peered":"2024-09-10T22:56:16.898240+0000","last_unstale":"2024-09-10T22:57:27.464902+0000","last_undegraded":"2024-09-10T22:57:27.464902+0000","last_fullsized":"2024-09-10T22:57:27.464902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:27.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267831+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:56:31.267831+0000","last_peered":"2024-09-10T22:56:31.267831+0000","last_clean":"2024-09-10T22:56:31.267831+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:56:31.267831+0000","last_undegraded":"2024-09-10T22:56:31.267831+0000","last_fullsized":"2024-09-10T22:56:31.267831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268468+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:56:31.268468+0000","last_peered":"2024-09-10T22:56:31.268468+0000","last_clean":"2024-09-10T22:56:31.268468+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:56:31.268468+0000","last_undegraded":"2024-09-10T22:56:31.268468+0000","last_fullsized":"2024-09-10T22:56:31.268468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215229+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:56:31.215229+0000","last_peered":"2024-09-10T22:56:31.215229+0000","last_clean":"2024-09-10T22:56:31.215229+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:56:31.215229+0000","last_undegraded":"2024-09-10T22:56:31.215229+0000","last_fullsized":"2024-09-10T22:56:31.215229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268206+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:56:31.268206+0000","last_peered":"2024-09-10T22:56:31.268206+0000","last_clean":"2024-09-10T22:56:31.268206+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:56:31.268206+0000","last_undegraded":"2024-09-10T22:56:31.268206+0000","last_fullsized":"2024-09-10T22:56:31.268206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268536+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:56:31.268536+0000","last_peered":"2024-09-10T22:56:31.268536+0000","last_clean":"2024-09-10T22:56:31.268536+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:56:31.268536+0000","last_undegraded":"2024-09-10T22:56:31.268536+0000","last_fullsized":"2024-09-10T22:56:31.268536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269565+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:56:31.269565+0000","last_peered":"2024-09-10T22:56:31.269565+0000","last_clean":"2024-09-10T22:56:31.269565+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:56:31.269565+0000","last_undegraded":"2024-09-10T22:56:31.269565+0000","last_fullsized":"2024-09-10T22:56:31.269565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973405+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:56:30.973405+0000","last_peered":"2024-09-10T22:56:30.973405+0000","last_clean":"2024-09-10T22:56:30.973405+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:56:30.973405+0000","last_undegraded":"2024-09-10T22:56:30.973405+0000","last_fullsized":"2024-09-10T22:56:30.973405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215537+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:56:31.215537+0000","last_peered":"2024-09-10T22:56:31.215537+0000","last_clean":"2024-09-10T22:56:31.215537+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:56:31.215537+0000","last_undegraded":"2024-09-10T22:56:31.215537+0000","last_fullsized":"2024-09-10T22:56:31.215537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268394+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:56:31.268394+0000","last_peered":"2024-09-10T22:56:31.268394+0000","last_clean":"2024-09-10T22:56:31.268394+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:56:31.268394+0000","last_undegraded":"2024-09-10T22:56:31.268394+0000","last_fullsized":"2024-09-10T22:56:31.268394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269581+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:56:31.269581+0000","last_peered":"2024-09-10T22:56:31.269581+0000","last_clean":"2024-09-10T22:56:31.269581+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:56:31.269581+0000","last_undegraded":"2024-09-10T22:56:31.269581+0000","last_fullsized":"2024-09-10T22:56:31.269581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974439+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:56:30.974439+0000","last_peered":"2024-09-10T22:56:30.974439+0000","last_clean":"2024-09-10T22:56:30.974439+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:56:30.974439+0000","last_undegraded":"2024-09-10T22:56:30.974439+0000","last_fullsized":"2024-09-10T22:56:30.974439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268950+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:56:31.268950+0000","last_peered":"2024-09-10T22:56:31.268950+0000","last_clean":"2024-09-10T22:56:31.268950+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:56:31.268950+0000","last_undegraded":"2024-09-10T22:56:31.268950+0000","last_fullsized":"2024-09-10T22:56:31.268950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974751+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:56:30.974751+0000","last_peered":"2024-09-10T22:56:30.974751+0000","last_clean":"2024-09-10T22:56:30.974751+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:56:30.974751+0000","last_undegraded":"2024-09-10T22:56:30.974751+0000","last_fullsized":"2024-09-10T22:56:30.974751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973265+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:56:30.973265+0000","last_peered":"2024-09-10T22:56:30.973265+0000","last_clean":"2024-09-10T22:56:30.973265+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:56:30.973265+0000","last_undegraded":"2024-09-10T22:56:30.973265+0000","last_fullsized":"2024-09-10T22:56:30.973265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215368+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:56:31.215368+0000","last_peered":"2024-09-10T22:56:31.215368+0000","last_clean":"2024-09-10T22:56:31.215368+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:56:31.215368+0000","last_undegraded":"2024-09-10T22:56:31.215368+0000","last_fullsized":"2024-09-10T22:56:31.215368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268566+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:56:31.268566+0000","last_peered":"2024-09-10T22:56:31.268566+0000","last_clean":"2024-09-10T22:56:31.268566+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:56:31.268566+0000","last_undegraded":"2024-09-10T22:56:31.268566+0000","last_fullsized":"2024-09-10T22:56:31.268566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270668+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:56:31.270668+0000","last_peered":"2024-09-10T22:56:31.270668+0000","last_clean":"2024-09-10T22:56:31.270668+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:56:31.270668+0000","last_undegraded":"2024-09-10T22:56:31.270668+0000","last_fullsized":"2024-09-10T22:56:31.270668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215964+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:56:31.215964+0000","last_peered":"2024-09-10T22:56:31.215964+0000","last_clean":"2024-09-10T22:56:31.215964+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:56:31.215964+0000","last_undegraded":"2024-09-10T22:56:31.215964+0000","last_fullsized":"2024-09-10T22:56:31.215964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974107+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:56:30.974107+0000","last_peered":"2024-09-10T22:56:30.974107+0000","last_clean":"2024-09-10T22:56:30.974107+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:56:30.974107+0000","last_undegraded":"2024-09-10T22:56:30.974107+0000","last_fullsized":"2024-09-10T22:56:30.974107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268385+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:56:31.268385+0000","last_peered":"2024-09-10T22:56:31.268385+0000","last_clean":"2024-09-10T22:56:31.268385+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:56:31.268385+0000","last_undegraded":"2024-09-10T22:56:31.268385+0000","last_fullsized":"2024-09-10T22:56:31.268385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268878+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:56:31.268878+0000","last_peered":"2024-09-10T22:56:31.268878+0000","last_clean":"2024-09-10T22:56:31.268878+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:56:31.268878+0000","last_undegraded":"2024-09-10T22:56:31.268878+0000","last_fullsized":"2024-09-10T22:56:31.268878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973261+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:56:30.973261+0000","last_peered":"2024-09-10T22:56:30.973261+0000","last_clean":"2024-09-10T22:56:30.973261+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:56:30.973261+0000","last_undegraded":"2024-09-10T22:56:30.973261+0000","last_fullsized":"2024-09-10T22:56:30.973261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114958+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:56:31.114958+0000","last_peered":"2024-09-10T22:56:31.114958+0000","last_clean":"2024-09-10T22:56:31.114958+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:56:31.114958+0000","last_undegraded":"2024-09-10T22:56:31.114958+0000","last_fullsized":"2024-09-10T22:56:31.114958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269181+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:56:31.269181+0000","last_peered":"2024-09-10T22:56:31.269181+0000","last_clean":"2024-09-10T22:56:31.269181+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:56:31.269181+0000","last_undegraded":"2024-09-10T22:56:31.269181+0000","last_fullsized":"2024-09-10T22:56:31.269181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269499+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:56:31.269499+0000","last_peered":"2024-09-10T22:56:31.269499+0000","last_clean":"2024-09-10T22:56:31.269499+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:56:31.269499+0000","last_undegraded":"2024-09-10T22:56:31.269499+0000","last_fullsized":"2024-09-10T22:56:31.269499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.974796+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:56:30.974796+0000","last_peered":"2024-09-10T22:56:30.974796+0000","last_clean":"2024-09-10T22:56:30.974796+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:56:30.974796+0000","last_undegraded":"2024-09-10T22:56:30.974796+0000","last_fullsized":"2024-09-10T22:56:30.974796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268808+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:56:31.268808+0000","last_peered":"2024-09-10T22:56:31.268808+0000","last_clean":"2024-09-10T22:56:31.268808+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:56:31.268808+0000","last_undegraded":"2024-09-10T22:56:31.268808+0000","last_fullsized":"2024-09-10T22:56:31.268808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268212+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:56:31.268212+0000","last_peered":"2024-09-10T22:56:31.268212+0000","last_clean":"2024-09-10T22:56:31.268212+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:56:31.268212+0000","last_undegraded":"2024-09-10T22:56:31.268212+0000","last_fullsized":"2024-09-10T22:56:31.268212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268610+0000","last_change":"2024-09-10T22:56:18.924551+0000","last_active":"2024-09-10T22:56:31.268610+0000","last_peered":"2024-09-10T22:56:31.268610+0000","last_clean":"2024-09-10T22:56:31.268610+0000","last_became_active":"2024-09-10T22:56:18.923067+0000","last_became_peered":"2024-09-10T22:56:18.923067+0000","last_unstale":"2024-09-10T22:56:31.268610+0000","last_undegraded":"2024-09-10T22:56:31.268610+0000","last_fullsized":"2024-09-10T22:56:31.268610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:55.514830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267628+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:56:31.267628+0000","last_peered":"2024-09-10T22:56:31.267628+0000","last_clean":"2024-09-10T22:56:31.267628+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:56:31.267628+0000","last_undegraded":"2024-09-10T22:56:31.267628+0000","last_fullsized":"2024-09-10T22:56:31.267628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114833+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:56:31.114833+0000","last_peered":"2024-09-10T22:56:31.114833+0000","last_clean":"2024-09-10T22:56:31.114833+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:56:31.114833+0000","last_undegraded":"2024-09-10T22:56:31.114833+0000","last_fullsized":"2024-09-10T22:56:31.114833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269828+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:56:31.269828+0000","last_peered":"2024-09-10T22:56:31.269828+0000","last_clean":"2024-09-10T22:56:31.269828+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:56:31.269828+0000","last_undegraded":"2024-09-10T22:56:31.269828+0000","last_fullsized":"2024-09-10T22:56:31.269828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269912+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:56:31.269912+0000","last_peered":"2024-09-10T22:56:31.269912+0000","last_clean":"2024-09-10T22:56:31.269912+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:56:31.269912+0000","last_undegraded":"2024-09-10T22:56:31.269912+0000","last_fullsized":"2024-09-10T22:56:31.269912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.215879+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:56:31.215879+0000","last_peered":"2024-09-10T22:56:31.215879+0000","last_clean":"2024-09-10T22:56:31.215879+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:56:31.215879+0000","last_undegraded":"2024-09-10T22:56:31.215879+0000","last_fullsized":"2024-09-10T22:56:31.215879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56405,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":135,"num_read_kb":129,"num_write":162,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":479658,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1119330,"data_compressed":53559,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":483990,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":479814,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1075723,"data_compressed":53559,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":475070,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1071311,"data_compressed":53559,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":479720,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117068,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":475264,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":529125,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":527384,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":476458,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:57:34.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:57:34.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:34 smithi067 ceph-mon[30939]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:34.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:34 smithi067 ceph-mon[30939]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:57:34.686 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:34.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:34 smithi092 ceph-mon[34223]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:34.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:34 smithi092 ceph-mon[34223]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:57:34.782 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:34 smithi145 ceph-mon[34397]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:34.782 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:34 smithi145 ceph-mon[34397]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.251+0000 7f4379e00640 1 -- 172.21.15.67:0/270458454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f437410f3c0 msgr2=0x7f43741117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.251+0000 7f4379e00640 1 --2- 172.21.15.67:0/270458454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f437410f3c0 0x7f43741117b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ca0 tx=0x7f436c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 -- 172.21.15.67:0/270458454 shutdown_connections 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 --2- 172.21.15.67:0/270458454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f437410f3c0 0x7f43741117b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 --2- 172.21.15.67:0/270458454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4374075f40 0x7f437410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 --2- 172.21.15.67:0/270458454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075540 0x7f4374075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 -- 172.21.15.67:0/270458454 >> 172.21.15.67:0/270458454 conn(0x7f43740fe810 msgr2=0x7f4374100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 -- 172.21.15.67:0/270458454 shutdown_connections 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 -- 172.21.15.67:0/270458454 wait complete. 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.252+0000 7f4379e00640 1 Processor -- start 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 -- start start 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075f40 0x7f437419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f437410f3c0 0x7f43741a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4374114e60 con 0x7f4374075540 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4374114ce0 con 0x7f437410f3c0 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4379e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4374114fe0 con 0x7f4374075f40 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4372a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:35.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4372a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46240/0 (socket says 172.21.15.67:46240) 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4372a00640 1 -- 172.21.15.67:0/2524888501 learned_addr learned my addr 172.21.15.67:0/2524888501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f4373400640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f437410f3c0 0x7f43741a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.253+0000 7f436a000640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075f40 0x7f437419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 -- 172.21.15.67:0/2524888501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075f40 msgr2=0x7f437419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075f40 0x7f437419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 -- 172.21.15.67:0/2524888501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f437410f3c0 msgr2=0x7f43741a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f437410f3c0 0x7f43741a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 -- 172.21.15.67:0/2524888501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741a1460 con 0x7f4374075540 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4372a00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f436400d770 tx=0x7f436400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364017070 con 0x7f4374075540 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43640054b0 con 0x7f4374075540 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741a1750 con 0x7f4374075540 2024-09-10T22:57:35.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.254+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364004c60 con 0x7f4374075540 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.255+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741a7e10 con 0x7f4374075540 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.256+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374075920 con 0x7f4374075540 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.258+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4364004e00 con 0x7f4374075540 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.259+0000 7f4378a00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 0x7f43580b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.260+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f43640e5c00 con 0x7f4374075540 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.260+0000 7f436a000640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 0x7f43580b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.260+0000 7f436a000640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 0x7f43580b7880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4374111a30 tx=0x7f435c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:35.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.263+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f43640abe90 con 0x7f4374075540 2024-09-10T22:57:35.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.456+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4374100b30 con 0x7f43580b53c0 2024-09-10T22:57:35.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.463+0000 7f4378a00640 1 -- 172.21.15.67:0/2524888501 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340607 (secure 0 0 0) 0x7f4374100b30 con 0x7f43580b53c0 2024-09-10T22:57:35.468 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 msgr2=0x7f43580b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 0x7f43580b7880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4374111a30 tx=0x7f435c00b410 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 msgr2=0x7f437419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f436400d770 tx=0x7f436400dc30 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 shutdown_connections 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43580b53c0 0x7f43580b7880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f437410f3c0 0x7f43741a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4374075f40 0x7f437419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 --2- 172.21.15.67:0/2524888501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4374075540 0x7f437419b3b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.467+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 >> 172.21.15.67:0/2524888501 conn(0x7f43740fe810 msgr2=0x7f43740ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:35.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.468+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 shutdown_connections 2024-09-10T22:57:35.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:35.468+0000 7f4379e00640 1 -- 172.21.15.67:0/2524888501 wait complete. 2024-09-10T22:57:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:35 smithi092 ceph-mon[34223]: pgmap v346: 129 pgs: 1 activating, 2 peering, 126 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:35 smithi092 ceph-mon[34223]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:57:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:35 smithi067 ceph-mon[30939]: pgmap v346: 129 pgs: 1 activating, 2 peering, 126 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:35 smithi067 ceph-mon[30939]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:57:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:35 smithi145 ceph-mon[34397]: pgmap v346: 129 pgs: 1 activating, 2 peering, 126 active+clean; 632 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:35 smithi145 ceph-mon[34397]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:57:36.443 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T22:57:34.075357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647007,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":216,"num_read_kb":197,"num_write":287,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2952853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356552,"kb_used_data":11440,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585272,"statfs":{"total":1151940427776,"available":1151575318528,"internally_reserved":0,"allocated":11714560,"data_stored":7739678,"data_compressed":337835,"data_compressed_allocated":454656,"data_compressed_original":2408448,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9180,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":23,"num_write":16,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002137"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425984+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:10.425984+0000","last_peered":"2024-09-10T22:57:10.425984+0000","last_clean":"2024-09-10T22:57:10.425984+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:10.425984+0000","last_undegraded":"2024-09-10T22:57:10.425984+0000","last_fullsized":"2024-09-10T22:57:10.425984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483317+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:57:33.483317+0000","last_peered":"2024-09-10T22:57:33.483317+0000","last_clean":"2024-09-10T22:57:33.483317+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:57:33.483317+0000","last_undegraded":"2024-09-10T22:57:33.483317+0000","last_fullsized":"2024-09-10T22:57:33.483317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476402+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:57:32.476402+0000","last_peered":"2024-09-10T22:57:32.476402+0000","last_clean":"2024-09-10T22:57:32.476402+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:57:32.476402+0000","last_undegraded":"2024-09-10T22:57:32.476402+0000","last_fullsized":"2024-09-10T22:57:32.476402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483508+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:57:33.483508+0000","last_peered":"2024-09-10T22:57:33.483508+0000","last_clean":"2024-09-10T22:57:33.483508+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:57:33.483508+0000","last_undegraded":"2024-09-10T22:57:33.483508+0000","last_fullsized":"2024-09-10T22:57:33.483508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270294+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:56:31.270294+0000","last_peered":"2024-09-10T22:56:31.270294+0000","last_clean":"2024-09-10T22:56:31.270294+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:56:31.270294+0000","last_undegraded":"2024-09-10T22:56:31.270294+0000","last_fullsized":"2024-09-10T22:56:31.270294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114275+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:56:31.114275+0000","last_peered":"2024-09-10T22:56:31.114275+0000","last_clean":"2024-09-10T22:56:31.114275+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:56:31.114275+0000","last_undegraded":"2024-09-10T22:56:31.114275+0000","last_fullsized":"2024-09-10T22:56:31.114275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.423890+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:10.423890+0000","last_peered":"2024-09-10T22:57:10.423890+0000","last_clean":"2024-09-10T22:57:10.423890+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:10.423890+0000","last_undegraded":"2024-09-10T22:57:10.423890+0000","last_fullsized":"2024-09-10T22:57:10.423890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484689+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:57:33.484689+0000","last_peered":"2024-09-10T22:57:33.484689+0000","last_clean":"2024-09-10T22:57:33.484689+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:57:33.484689+0000","last_undegraded":"2024-09-10T22:57:33.484689+0000","last_fullsized":"2024-09-10T22:57:33.484689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114523+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:56:31.114523+0000","last_peered":"2024-09-10T22:56:31.114523+0000","last_clean":"2024-09-10T22:56:31.114523+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:56:31.114523+0000","last_undegraded":"2024-09-10T22:56:31.114523+0000","last_fullsized":"2024-09-10T22:56:31.114523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-10T22:56:19.912821+0000","last_change":"2024-09-10T22:57:32.903687+0000","last_active":"2024-09-10T22:57:32.903687+0000","last_peered":"2024-09-10T22:56:19.912821+0000","last_clean":"2024-09-10T22:56:19.912821+0000","last_became_active":"2024-09-10T22:56:16.890866+0000","last_became_peered":"2024-09-10T22:56:16.890866+0000","last_unstale":"2024-09-10T22:56:19.912821+0000","last_undegraded":"2024-09-10T22:56:19.912821+0000","last_fullsized":"2024-09-10T22:56:19.912821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"activating","last_fresh":"2024-09-10T22:57:33.485317+0000","last_change":"2024-09-10T22:57:33.485317+0000","last_active":"2024-09-10T22:57:32.482204+0000","last_peered":"2024-09-10T22:56:30.975209+0000","last_clean":"2024-09-10T22:56:30.975209+0000","last_became_active":"2024-09-10T22:56:18.921985+0000","last_became_peered":"2024-09-10T22:56:18.921985+0000","last_unstale":"2024-09-10T22:57:33.485317+0000","last_undegraded":"2024-09-10T22:57:33.485317+0000","last_fullsized":"2024-09-10T22:57:33.485317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":["5","1","4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114411+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:56:31.114411+0000","last_peered":"2024-09-10T22:56:31.114411+0000","last_clean":"2024-09-10T22:56:31.114411+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:56:31.114411+0000","last_undegraded":"2024-09-10T22:56:31.114411+0000","last_fullsized":"2024-09-10T22:56:31.114411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489400+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:57:33.489400+0000","last_peered":"2024-09-10T22:57:33.489400+0000","last_clean":"2024-09-10T22:57:33.489400+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:57:33.489400+0000","last_undegraded":"2024-09-10T22:57:33.489400+0000","last_fullsized":"2024-09-10T22:57:33.489400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.416212+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:09.416212+0000","last_peered":"2024-09-10T22:57:09.416212+0000","last_clean":"2024-09-10T22:57:09.416212+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:09.416212+0000","last_undegraded":"2024-09-10T22:57:09.416212+0000","last_fullsized":"2024-09-10T22:57:09.416212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114671+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:56:31.114671+0000","last_peered":"2024-09-10T22:56:31.114671+0000","last_clean":"2024-09-10T22:56:31.114671+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:56:31.114671+0000","last_undegraded":"2024-09-10T22:56:31.114671+0000","last_fullsized":"2024-09-10T22:56:31.114671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483072+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:57:33.483072+0000","last_peered":"2024-09-10T22:57:33.483072+0000","last_clean":"2024-09-10T22:57:33.483072+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:57:33.483072+0000","last_undegraded":"2024-09-10T22:57:33.483072+0000","last_fullsized":"2024-09-10T22:57:33.483072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476456+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:57:32.476456+0000","last_peered":"2024-09-10T22:57:32.476456+0000","last_clean":"2024-09-10T22:57:32.476456+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:57:32.476456+0000","last_undegraded":"2024-09-10T22:57:32.476456+0000","last_fullsized":"2024-09-10T22:57:32.476456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114845+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:56:31.114845+0000","last_peered":"2024-09-10T22:56:31.114845+0000","last_clean":"2024-09-10T22:56:31.114845+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:56:31.114845+0000","last_undegraded":"2024-09-10T22:56:31.114845+0000","last_fullsized":"2024-09-10T22:56:31.114845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.483016+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:57:32.483016+0000","last_peered":"2024-09-10T22:57:32.483016+0000","last_clean":"2024-09-10T22:57:32.483016+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:57:32.483016+0000","last_undegraded":"2024-09-10T22:57:32.483016+0000","last_fullsized":"2024-09-10T22:57:32.483016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482999+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:33.482999+0000","last_peered":"2024-09-10T22:57:33.482999+0000","last_clean":"2024-09-10T22:57:33.482999+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:33.482999+0000","last_undegraded":"2024-09-10T22:57:33.482999+0000","last_fullsized":"2024-09-10T22:57:33.482999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269937+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:56:31.269937+0000","last_peered":"2024-09-10T22:56:31.269937+0000","last_clean":"2024-09-10T22:56:31.269937+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:56:31.269937+0000","last_undegraded":"2024-09-10T22:56:31.269937+0000","last_fullsized":"2024-09-10T22:56:31.269937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267439+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:56:31.267439+0000","last_peered":"2024-09-10T22:56:31.267439+0000","last_clean":"2024-09-10T22:56:31.267439+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:56:31.267439+0000","last_undegraded":"2024-09-10T22:56:31.267439+0000","last_fullsized":"2024-09-10T22:56:31.267439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484357+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:33.484357+0000","last_peered":"2024-09-10T22:57:33.484357+0000","last_clean":"2024-09-10T22:57:33.484357+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:33.484357+0000","last_undegraded":"2024-09-10T22:57:33.484357+0000","last_fullsized":"2024-09-10T22:57:33.484357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114846+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:56:31.114846+0000","last_peered":"2024-09-10T22:56:31.114846+0000","last_clean":"2024-09-10T22:56:31.114846+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:56:31.114846+0000","last_undegraded":"2024-09-10T22:56:31.114846+0000","last_fullsized":"2024-09-10T22:56:31.114846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270007+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:56:31.270007+0000","last_peered":"2024-09-10T22:56:31.270007+0000","last_clean":"2024-09-10T22:56:31.270007+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:56:31.270007+0000","last_undegraded":"2024-09-10T22:56:31.270007+0000","last_fullsized":"2024-09-10T22:56:31.270007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.414042+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:09.414042+0000","last_peered":"2024-09-10T22:57:09.414042+0000","last_clean":"2024-09-10T22:57:09.414042+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:09.414042+0000","last_undegraded":"2024-09-10T22:57:09.414042+0000","last_fullsized":"2024-09-10T22:57:09.414042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477032+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:57:32.477032+0000","last_peered":"2024-09-10T22:57:32.477032+0000","last_clean":"2024-09-10T22:57:32.477032+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:57:32.477032+0000","last_undegraded":"2024-09-10T22:57:32.477032+0000","last_fullsized":"2024-09-10T22:57:32.477032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483447+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:57:33.483447+0000","last_peered":"2024-09-10T22:57:33.483447+0000","last_clean":"2024-09-10T22:57:33.483447+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:57:33.483447+0000","last_undegraded":"2024-09-10T22:57:33.483447+0000","last_fullsized":"2024-09-10T22:57:33.483447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.916037+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:27.916037+0000","last_peered":"2024-09-10T22:57:27.916037+0000","last_clean":"2024-09-10T22:57:27.916037+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:27.916037+0000","last_undegraded":"2024-09-10T22:57:27.916037+0000","last_fullsized":"2024-09-10T22:57:27.916037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482709+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:57:33.482709+0000","last_peered":"2024-09-10T22:57:33.482709+0000","last_clean":"2024-09-10T22:57:33.482709+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:57:33.482709+0000","last_undegraded":"2024-09-10T22:57:33.482709+0000","last_fullsized":"2024-09-10T22:57:33.482709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973506+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:56:30.973506+0000","last_peered":"2024-09-10T22:56:30.973506+0000","last_clean":"2024-09-10T22:56:30.973506+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:56:30.973506+0000","last_undegraded":"2024-09-10T22:56:30.973506+0000","last_fullsized":"2024-09-10T22:56:30.973506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:09.413823+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:09.413823+0000","last_peered":"2024-09-10T22:57:09.413823+0000","last_clean":"2024-09-10T22:57:09.413823+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:09.413823+0000","last_undegraded":"2024-09-10T22:57:09.413823+0000","last_fullsized":"2024-09-10T22:57:09.413823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.482103+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:32.482103+0000","last_peered":"2024-09-10T22:57:32.482103+0000","last_clean":"2024-09-10T22:57:32.482103+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:32.482103+0000","last_undegraded":"2024-09-10T22:57:32.482103+0000","last_fullsized":"2024-09-10T22:57:32.482103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268840+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:56:31.268840+0000","last_peered":"2024-09-10T22:56:31.268840+0000","last_clean":"2024-09-10T22:56:31.268840+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:56:31.268840+0000","last_undegraded":"2024-09-10T22:56:31.268840+0000","last_fullsized":"2024-09-10T22:56:31.268840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269684+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:56:31.269684+0000","last_peered":"2024-09-10T22:56:31.269684+0000","last_clean":"2024-09-10T22:56:31.269684+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:56:31.269684+0000","last_undegraded":"2024-09-10T22:56:31.269684+0000","last_fullsized":"2024-09-10T22:56:31.269684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267571+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:56:31.267571+0000","last_peered":"2024-09-10T22:56:31.267571+0000","last_clean":"2024-09-10T22:56:31.267571+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:56:31.267571+0000","last_undegraded":"2024-09-10T22:56:31.267571+0000","last_fullsized":"2024-09-10T22:56:31.267571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268978+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:56:31.268978+0000","last_peered":"2024-09-10T22:56:31.268978+0000","last_clean":"2024-09-10T22:56:31.268978+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:56:31.268978+0000","last_undegraded":"2024-09-10T22:56:31.268978+0000","last_fullsized":"2024-09-10T22:56:31.268978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489599+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:57:33.489599+0000","last_peered":"2024-09-10T22:57:33.489599+0000","last_clean":"2024-09-10T22:57:33.489599+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:57:33.489599+0000","last_undegraded":"2024-09-10T22:57:33.489599+0000","last_fullsized":"2024-09-10T22:57:33.489599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973373+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:56:30.973373+0000","last_peered":"2024-09-10T22:56:30.973373+0000","last_clean":"2024-09-10T22:56:30.973373+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:56:30.973373+0000","last_undegraded":"2024-09-10T22:56:30.973373+0000","last_fullsized":"2024-09-10T22:56:30.973373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484247+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:57:33.484247+0000","last_peered":"2024-09-10T22:57:33.484247+0000","last_clean":"2024-09-10T22:57:33.484247+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:57:33.484247+0000","last_undegraded":"2024-09-10T22:57:33.484247+0000","last_fullsized":"2024-09-10T22:57:33.484247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:28.466035+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:57:28.466035+0000","last_peered":"2024-09-10T22:57:28.466035+0000","last_clean":"2024-09-10T22:57:28.466035+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:57:28.466035+0000","last_undegraded":"2024-09-10T22:57:28.466035+0000","last_fullsized":"2024-09-10T22:57:28.466035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973586+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:56:30.973586+0000","last_peered":"2024-09-10T22:56:30.973586+0000","last_clean":"2024-09-10T22:56:30.973586+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:56:30.973586+0000","last_undegraded":"2024-09-10T22:56:30.973586+0000","last_fullsized":"2024-09-10T22:56:30.973586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.972886+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:56:30.972886+0000","last_peered":"2024-09-10T22:56:30.972886+0000","last_clean":"2024-09-10T22:56:30.972886+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:56:30.972886+0000","last_undegraded":"2024-09-10T22:56:30.972886+0000","last_fullsized":"2024-09-10T22:56:30.972886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487480+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:33.487480+0000","last_peered":"2024-09-10T22:57:33.487480+0000","last_clean":"2024-09-10T22:57:33.487480+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:33.487480+0000","last_undegraded":"2024-09-10T22:57:33.487480+0000","last_fullsized":"2024-09-10T22:57:33.487480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484718+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:57:33.484718+0000","last_peered":"2024-09-10T22:57:33.484718+0000","last_clean":"2024-09-10T22:57:33.484718+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:57:33.484718+0000","last_undegraded":"2024-09-10T22:57:33.484718+0000","last_fullsized":"2024-09-10T22:57:33.484718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269073+0000","last_change":"2024-09-10T22:56:18.920310+0000","last_active":"2024-09-10T22:56:31.269073+0000","last_peered":"2024-09-10T22:56:31.269073+0000","last_clean":"2024-09-10T22:56:31.269073+0000","last_became_active":"2024-09-10T22:56:18.918860+0000","last_became_peered":"2024-09-10T22:56:18.918860+0000","last_unstale":"2024-09-10T22:56:31.269073+0000","last_undegraded":"2024-09-10T22:56:31.269073+0000","last_fullsized":"2024-09-10T22:56:31.269073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:33.506508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:12.434910+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:12.434910+0000","last_peered":"2024-09-10T22:57:12.434910+0000","last_clean":"2024-09-10T22:57:12.434910+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:12.434910+0000","last_undegraded":"2024-09-10T22:57:12.434910+0000","last_fullsized":"2024-09-10T22:57:12.434910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487351+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:57:33.487351+0000","last_peered":"2024-09-10T22:57:33.487351+0000","last_clean":"2024-09-10T22:57:33.487351+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:57:33.487351+0000","last_undegraded":"2024-09-10T22:57:33.487351+0000","last_fullsized":"2024-09-10T22:57:33.487351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487434+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:57:33.487434+0000","last_peered":"2024-09-10T22:57:33.487434+0000","last_clean":"2024-09-10T22:57:33.487434+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:57:33.487434+0000","last_undegraded":"2024-09-10T22:57:33.487434+0000","last_fullsized":"2024-09-10T22:57:33.487434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487965+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:57:33.487965+0000","last_peered":"2024-09-10T22:57:33.487965+0000","last_clean":"2024-09-10T22:57:33.487965+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:57:33.487965+0000","last_undegraded":"2024-09-10T22:57:33.487965+0000","last_fullsized":"2024-09-10T22:57:33.487965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477275+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:32.477275+0000","last_peered":"2024-09-10T22:57:32.477275+0000","last_clean":"2024-09-10T22:57:32.477275+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:32.477275+0000","last_undegraded":"2024-09-10T22:57:32.477275+0000","last_fullsized":"2024-09-10T22:57:32.477275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484515+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:57:33.484515+0000","last_peered":"2024-09-10T22:57:33.484515+0000","last_clean":"2024-09-10T22:57:33.484515+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:57:33.484515+0000","last_undegraded":"2024-09-10T22:57:33.484515+0000","last_fullsized":"2024-09-10T22:57:33.484515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495905+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:57:33.495905+0000","last_peered":"2024-09-10T22:57:33.495905+0000","last_clean":"2024-09-10T22:57:33.495905+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:57:33.495905+0000","last_undegraded":"2024-09-10T22:57:33.495905+0000","last_fullsized":"2024-09-10T22:57:33.495905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485826+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:57:33.485826+0000","last_peered":"2024-09-10T22:57:33.485826+0000","last_clean":"2024-09-10T22:57:33.485826+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:57:33.485826+0000","last_undegraded":"2024-09-10T22:57:33.485826+0000","last_fullsized":"2024-09-10T22:57:33.485826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269468+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:56:31.269468+0000","last_peered":"2024-09-10T22:56:31.269468+0000","last_clean":"2024-09-10T22:56:31.269468+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:56:31.269468+0000","last_undegraded":"2024-09-10T22:56:31.269468+0000","last_fullsized":"2024-09-10T22:56:31.269468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487627+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:57:33.487627+0000","last_peered":"2024-09-10T22:57:33.487627+0000","last_clean":"2024-09-10T22:57:33.487627+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:57:33.487627+0000","last_undegraded":"2024-09-10T22:57:33.487627+0000","last_fullsized":"2024-09-10T22:57:33.487627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973510+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:56:30.973510+0000","last_peered":"2024-09-10T22:56:30.973510+0000","last_clean":"2024-09-10T22:56:30.973510+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:56:30.973510+0000","last_undegraded":"2024-09-10T22:56:30.973510+0000","last_fullsized":"2024-09-10T22:56:30.973510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483887+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:57:33.483887+0000","last_peered":"2024-09-10T22:57:33.483887+0000","last_clean":"2024-09-10T22:57:33.483887+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:57:33.483887+0000","last_undegraded":"2024-09-10T22:57:33.483887+0000","last_fullsized":"2024-09-10T22:57:33.483887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484195+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:57:32.484195+0000","last_peered":"2024-09-10T22:57:32.484195+0000","last_clean":"2024-09-10T22:57:32.484195+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:57:32.484195+0000","last_undegraded":"2024-09-10T22:57:32.484195+0000","last_fullsized":"2024-09-10T22:57:32.484195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:11.418870+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:11.418870+0000","last_peered":"2024-09-10T22:57:11.418870+0000","last_clean":"2024-09-10T22:57:11.418870+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:11.418870+0000","last_undegraded":"2024-09-10T22:57:11.418870+0000","last_fullsized":"2024-09-10T22:57:11.418870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484068+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:57:33.484068+0000","last_peered":"2024-09-10T22:57:33.484068+0000","last_clean":"2024-09-10T22:57:33.484068+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:57:33.484068+0000","last_undegraded":"2024-09-10T22:57:33.484068+0000","last_fullsized":"2024-09-10T22:57:33.484068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490385+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:57:33.490385+0000","last_peered":"2024-09-10T22:57:33.490385+0000","last_clean":"2024-09-10T22:57:33.490385+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:57:33.490385+0000","last_undegraded":"2024-09-10T22:57:33.490385+0000","last_fullsized":"2024-09-10T22:57:33.490385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-10T22:57:09.413906+0000","last_change":"2024-09-10T22:57:32.903638+0000","last_active":"2024-09-10T22:57:32.903638+0000","last_peered":"2024-09-10T22:57:09.413906+0000","last_clean":"2024-09-10T22:57:09.413906+0000","last_became_active":"2024-09-10T22:56:17.605996+0000","last_became_peered":"2024-09-10T22:56:17.605996+0000","last_unstale":"2024-09-10T22:57:09.413906+0000","last_undegraded":"2024-09-10T22:57:09.413906+0000","last_fullsized":"2024-09-10T22:57:09.413906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973575+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:56:30.973575+0000","last_peered":"2024-09-10T22:56:30.973575+0000","last_clean":"2024-09-10T22:56:30.973575+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:56:30.973575+0000","last_undegraded":"2024-09-10T22:56:30.973575+0000","last_fullsized":"2024-09-10T22:56:30.973575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491949+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:57:33.491949+0000","last_peered":"2024-09-10T22:57:33.491949+0000","last_clean":"2024-09-10T22:57:33.491949+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:57:33.491949+0000","last_undegraded":"2024-09-10T22:57:33.491949+0000","last_fullsized":"2024-09-10T22:57:33.491949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.464935+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:57:27.464935+0000","last_peered":"2024-09-10T22:57:27.464935+0000","last_clean":"2024-09-10T22:57:27.464935+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:57:27.464935+0000","last_undegraded":"2024-09-10T22:57:27.464935+0000","last_fullsized":"2024-09-10T22:57:27.464935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270217+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:56:31.270217+0000","last_peered":"2024-09-10T22:56:31.270217+0000","last_clean":"2024-09-10T22:56:31.270217+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:56:31.270217+0000","last_undegraded":"2024-09-10T22:56:31.270217+0000","last_fullsized":"2024-09-10T22:56:31.270217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487381+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:57:33.487381+0000","last_peered":"2024-09-10T22:57:33.487381+0000","last_clean":"2024-09-10T22:57:33.487381+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:57:33.487381+0000","last_undegraded":"2024-09-10T22:57:33.487381+0000","last_fullsized":"2024-09-10T22:57:33.487381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484551+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:33.484551+0000","last_peered":"2024-09-10T22:57:33.484551+0000","last_clean":"2024-09-10T22:57:33.484551+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:33.484551+0000","last_undegraded":"2024-09-10T22:57:33.484551+0000","last_fullsized":"2024-09-10T22:57:33.484551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484140+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:32.484140+0000","last_peered":"2024-09-10T22:57:32.484140+0000","last_clean":"2024-09-10T22:57:32.484140+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:32.484140+0000","last_undegraded":"2024-09-10T22:57:32.484140+0000","last_fullsized":"2024-09-10T22:57:32.484140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973005+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:56:30.973005+0000","last_peered":"2024-09-10T22:56:30.973005+0000","last_clean":"2024-09-10T22:56:30.973005+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:56:30.973005+0000","last_undegraded":"2024-09-10T22:56:30.973005+0000","last_fullsized":"2024-09-10T22:56:30.973005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476668+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:32.476668+0000","last_peered":"2024-09-10T22:57:32.476668+0000","last_clean":"2024-09-10T22:57:32.476668+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:32.476668+0000","last_undegraded":"2024-09-10T22:57:32.476668+0000","last_fullsized":"2024-09-10T22:57:32.476668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485286+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:57:33.485286+0000","last_peered":"2024-09-10T22:57:33.485286+0000","last_clean":"2024-09-10T22:57:33.485286+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:57:33.485286+0000","last_undegraded":"2024-09-10T22:57:33.485286+0000","last_fullsized":"2024-09-10T22:57:33.485286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268147+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:56:31.268147+0000","last_peered":"2024-09-10T22:56:31.268147+0000","last_clean":"2024-09-10T22:56:31.268147+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:56:31.268147+0000","last_undegraded":"2024-09-10T22:56:31.268147+0000","last_fullsized":"2024-09-10T22:56:31.268147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:30.477235+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:57:30.477235+0000","last_peered":"2024-09-10T22:57:30.477235+0000","last_clean":"2024-09-10T22:57:30.477235+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:57:30.477235+0000","last_undegraded":"2024-09-10T22:57:30.477235+0000","last_fullsized":"2024-09-10T22:57:30.477235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52404,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269170+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:56:31.269170+0000","last_peered":"2024-09-10T22:56:31.269170+0000","last_clean":"2024-09-10T22:56:31.269170+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:56:31.269170+0000","last_undegraded":"2024-09-10T22:56:31.269170+0000","last_fullsized":"2024-09-10T22:56:31.269170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484785+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:57:33.484785+0000","last_peered":"2024-09-10T22:57:33.484785+0000","last_clean":"2024-09-10T22:57:33.484785+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:57:33.484785+0000","last_undegraded":"2024-09-10T22:57:33.484785+0000","last_fullsized":"2024-09-10T22:57:33.484785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485645+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:33.485645+0000","last_peered":"2024-09-10T22:57:33.485645+0000","last_clean":"2024-09-10T22:57:33.485645+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:33.485645+0000","last_undegraded":"2024-09-10T22:57:33.485645+0000","last_fullsized":"2024-09-10T22:57:33.485645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114831+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:56:31.114831+0000","last_peered":"2024-09-10T22:56:31.114831+0000","last_clean":"2024-09-10T22:56:31.114831+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:56:31.114831+0000","last_undegraded":"2024-09-10T22:56:31.114831+0000","last_fullsized":"2024-09-10T22:56:31.114831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270319+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:56:31.270319+0000","last_peered":"2024-09-10T22:56:31.270319+0000","last_clean":"2024-09-10T22:56:31.270319+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:56:31.270319+0000","last_undegraded":"2024-09-10T22:56:31.270319+0000","last_fullsized":"2024-09-10T22:56:31.270319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490487+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:33.490487+0000","last_peered":"2024-09-10T22:57:33.490487+0000","last_clean":"2024-09-10T22:57:33.490487+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:33.490487+0000","last_undegraded":"2024-09-10T22:57:33.490487+0000","last_fullsized":"2024-09-10T22:57:33.490487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268692+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:56:31.268692+0000","last_peered":"2024-09-10T22:56:31.268692+0000","last_clean":"2024-09-10T22:56:31.268692+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:56:31.268692+0000","last_undegraded":"2024-09-10T22:56:31.268692+0000","last_fullsized":"2024-09-10T22:56:31.268692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485175+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:57:33.485175+0000","last_peered":"2024-09-10T22:57:33.485175+0000","last_clean":"2024-09-10T22:57:33.485175+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:57:33.485175+0000","last_undegraded":"2024-09-10T22:57:33.485175+0000","last_fullsized":"2024-09-10T22:57:33.485175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:29.467786+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:57:29.467786+0000","last_peered":"2024-09-10T22:57:29.467786+0000","last_clean":"2024-09-10T22:57:29.467786+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:57:29.467786+0000","last_undegraded":"2024-09-10T22:57:29.467786+0000","last_fullsized":"2024-09-10T22:57:29.467786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482848+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:57:33.482848+0000","last_peered":"2024-09-10T22:57:33.482848+0000","last_clean":"2024-09-10T22:57:33.482848+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:57:33.482848+0000","last_undegraded":"2024-09-10T22:57:33.482848+0000","last_fullsized":"2024-09-10T22:57:33.482848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269754+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:56:31.269754+0000","last_peered":"2024-09-10T22:56:31.269754+0000","last_clean":"2024-09-10T22:56:31.269754+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:56:31.269754+0000","last_undegraded":"2024-09-10T22:56:31.269754+0000","last_fullsized":"2024-09-10T22:56:31.269754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268563+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:56:31.268563+0000","last_peered":"2024-09-10T22:56:31.268563+0000","last_clean":"2024-09-10T22:56:31.268563+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:56:31.268563+0000","last_undegraded":"2024-09-10T22:56:31.268563+0000","last_fullsized":"2024-09-10T22:56:31.268563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484688+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:57:33.484688+0000","last_peered":"2024-09-10T22:57:33.484688+0000","last_clean":"2024-09-10T22:57:33.484688+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:57:33.484688+0000","last_undegraded":"2024-09-10T22:57:33.484688+0000","last_fullsized":"2024-09-10T22:57:33.484688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484099+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:32.484099+0000","last_peered":"2024-09-10T22:57:32.484099+0000","last_clean":"2024-09-10T22:57:32.484099+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:32.484099+0000","last_undegraded":"2024-09-10T22:57:32.484099+0000","last_fullsized":"2024-09-10T22:57:32.484099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425449+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:10.425449+0000","last_peered":"2024-09-10T22:57:10.425449+0000","last_clean":"2024-09-10T22:57:10.425449+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:10.425449+0000","last_undegraded":"2024-09-10T22:57:10.425449+0000","last_fullsized":"2024-09-10T22:57:10.425449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.426510+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:10.426510+0000","last_peered":"2024-09-10T22:57:10.426510+0000","last_clean":"2024-09-10T22:57:10.426510+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:10.426510+0000","last_undegraded":"2024-09-10T22:57:10.426510+0000","last_fullsized":"2024-09-10T22:57:10.426510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484446+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:57:33.484446+0000","last_peered":"2024-09-10T22:57:33.484446+0000","last_clean":"2024-09-10T22:57:33.484446+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:57:33.484446+0000","last_undegraded":"2024-09-10T22:57:33.484446+0000","last_fullsized":"2024-09-10T22:57:33.484446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476645+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:57:32.476645+0000","last_peered":"2024-09-10T22:57:32.476645+0000","last_clean":"2024-09-10T22:57:32.476645+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:57:32.476645+0000","last_undegraded":"2024-09-10T22:57:32.476645+0000","last_fullsized":"2024-09-10T22:57:32.476645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.497103+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:57:33.497103+0000","last_peered":"2024-09-10T22:57:33.497103+0000","last_clean":"2024-09-10T22:57:33.497103+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:57:33.497103+0000","last_undegraded":"2024-09-10T22:57:33.497103+0000","last_fullsized":"2024-09-10T22:57:33.497103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267831+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:56:31.267831+0000","last_peered":"2024-09-10T22:56:31.267831+0000","last_clean":"2024-09-10T22:56:31.267831+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:56:31.267831+0000","last_undegraded":"2024-09-10T22:56:31.267831+0000","last_fullsized":"2024-09-10T22:56:31.267831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476809+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:57:32.476809+0000","last_peered":"2024-09-10T22:57:32.476809+0000","last_clean":"2024-09-10T22:57:32.476809+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:57:32.476809+0000","last_undegraded":"2024-09-10T22:57:32.476809+0000","last_fullsized":"2024-09-10T22:57:32.476809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.482461+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:57:32.482461+0000","last_peered":"2024-09-10T22:57:32.482461+0000","last_clean":"2024-09-10T22:57:32.482461+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:57:32.482461+0000","last_undegraded":"2024-09-10T22:57:32.482461+0000","last_fullsized":"2024-09-10T22:57:32.482461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477147+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:57:32.477147+0000","last_peered":"2024-09-10T22:57:32.477147+0000","last_clean":"2024-09-10T22:57:32.477147+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:57:32.477147+0000","last_undegraded":"2024-09-10T22:57:32.477147+0000","last_fullsized":"2024-09-10T22:57:32.477147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476565+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:57:32.476565+0000","last_peered":"2024-09-10T22:57:32.476565+0000","last_clean":"2024-09-10T22:57:32.476565+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:57:32.476565+0000","last_undegraded":"2024-09-10T22:57:32.476565+0000","last_fullsized":"2024-09-10T22:57:32.476565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269565+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:56:31.269565+0000","last_peered":"2024-09-10T22:56:31.269565+0000","last_clean":"2024-09-10T22:56:31.269565+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:56:31.269565+0000","last_undegraded":"2024-09-10T22:56:31.269565+0000","last_fullsized":"2024-09-10T22:56:31.269565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973405+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:56:30.973405+0000","last_peered":"2024-09-10T22:56:30.973405+0000","last_clean":"2024-09-10T22:56:30.973405+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:56:30.973405+0000","last_undegraded":"2024-09-10T22:56:30.973405+0000","last_fullsized":"2024-09-10T22:56:30.973405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.483994+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:57:32.483994+0000","last_peered":"2024-09-10T22:57:32.483994+0000","last_clean":"2024-09-10T22:57:32.483994+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:57:32.483994+0000","last_undegraded":"2024-09-10T22:57:32.483994+0000","last_fullsized":"2024-09-10T22:57:32.483994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476977+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:57:32.476977+0000","last_peered":"2024-09-10T22:57:32.476977+0000","last_clean":"2024-09-10T22:57:32.476977+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:57:32.476977+0000","last_undegraded":"2024-09-10T22:57:32.476977+0000","last_fullsized":"2024-09-10T22:57:32.476977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483396+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:57:33.483396+0000","last_peered":"2024-09-10T22:57:33.483396+0000","last_clean":"2024-09-10T22:57:33.483396+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:57:33.483396+0000","last_undegraded":"2024-09-10T22:57:33.483396+0000","last_fullsized":"2024-09-10T22:57:33.483396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484099+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:57:33.484099+0000","last_peered":"2024-09-10T22:57:33.484099+0000","last_clean":"2024-09-10T22:57:33.484099+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:57:33.484099+0000","last_undegraded":"2024-09-10T22:57:33.484099+0000","last_fullsized":"2024-09-10T22:57:33.484099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482935+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:57:33.482935+0000","last_peered":"2024-09-10T22:57:33.482935+0000","last_clean":"2024-09-10T22:57:33.482935+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:57:33.482935+0000","last_undegraded":"2024-09-10T22:57:33.482935+0000","last_fullsized":"2024-09-10T22:57:33.482935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484667+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:57:33.484667+0000","last_peered":"2024-09-10T22:57:33.484667+0000","last_clean":"2024-09-10T22:57:33.484667+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:57:33.484667+0000","last_undegraded":"2024-09-10T22:57:33.484667+0000","last_fullsized":"2024-09-10T22:57:33.484667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973265+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:56:30.973265+0000","last_peered":"2024-09-10T22:56:30.973265+0000","last_clean":"2024-09-10T22:56:30.973265+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:56:30.973265+0000","last_undegraded":"2024-09-10T22:56:30.973265+0000","last_fullsized":"2024-09-10T22:56:30.973265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484014+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:57:32.484014+0000","last_peered":"2024-09-10T22:57:32.484014+0000","last_clean":"2024-09-10T22:57:32.484014+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:57:32.484014+0000","last_undegraded":"2024-09-10T22:57:32.484014+0000","last_fullsized":"2024-09-10T22:57:32.484014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477049+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:57:32.477049+0000","last_peered":"2024-09-10T22:57:32.477049+0000","last_clean":"2024-09-10T22:57:32.477049+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:57:32.477049+0000","last_undegraded":"2024-09-10T22:57:32.477049+0000","last_fullsized":"2024-09-10T22:57:32.477049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270668+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:56:31.270668+0000","last_peered":"2024-09-10T22:56:31.270668+0000","last_clean":"2024-09-10T22:56:31.270668+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:56:31.270668+0000","last_undegraded":"2024-09-10T22:56:31.270668+0000","last_fullsized":"2024-09-10T22:56:31.270668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484410+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:57:32.484410+0000","last_peered":"2024-09-10T22:57:32.484410+0000","last_clean":"2024-09-10T22:57:32.484410+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:57:32.484410+0000","last_undegraded":"2024-09-10T22:57:32.484410+0000","last_fullsized":"2024-09-10T22:57:32.484410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484224+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:57:33.484224+0000","last_peered":"2024-09-10T22:57:33.484224+0000","last_clean":"2024-09-10T22:57:33.484224+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:57:33.484224+0000","last_undegraded":"2024-09-10T22:57:33.484224+0000","last_fullsized":"2024-09-10T22:57:33.484224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268385+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:56:31.268385+0000","last_peered":"2024-09-10T22:56:31.268385+0000","last_clean":"2024-09-10T22:56:31.268385+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:56:31.268385+0000","last_undegraded":"2024-09-10T22:56:31.268385+0000","last_fullsized":"2024-09-10T22:56:31.268385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268878+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:56:31.268878+0000","last_peered":"2024-09-10T22:56:31.268878+0000","last_clean":"2024-09-10T22:56:31.268878+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:56:31.268878+0000","last_undegraded":"2024-09-10T22:56:31.268878+0000","last_fullsized":"2024-09-10T22:56:31.268878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:30.973261+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:56:30.973261+0000","last_peered":"2024-09-10T22:56:30.973261+0000","last_clean":"2024-09-10T22:56:30.973261+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:56:30.973261+0000","last_undegraded":"2024-09-10T22:56:30.973261+0000","last_fullsized":"2024-09-10T22:56:30.973261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114958+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:56:31.114958+0000","last_peered":"2024-09-10T22:56:31.114958+0000","last_clean":"2024-09-10T22:56:31.114958+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:56:31.114958+0000","last_undegraded":"2024-09-10T22:56:31.114958+0000","last_fullsized":"2024-09-10T22:56:31.114958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487645+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:57:33.487645+0000","last_peered":"2024-09-10T22:57:33.487645+0000","last_clean":"2024-09-10T22:57:33.487645+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:57:33.487645+0000","last_undegraded":"2024-09-10T22:57:33.487645+0000","last_fullsized":"2024-09-10T22:57:33.487645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269499+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:56:31.269499+0000","last_peered":"2024-09-10T22:56:31.269499+0000","last_clean":"2024-09-10T22:56:31.269499+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:56:31.269499+0000","last_undegraded":"2024-09-10T22:56:31.269499+0000","last_fullsized":"2024-09-10T22:56:31.269499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485308+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:57:33.485308+0000","last_peered":"2024-09-10T22:57:33.485308+0000","last_clean":"2024-09-10T22:57:33.485308+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:57:33.485308+0000","last_undegraded":"2024-09-10T22:57:33.485308+0000","last_fullsized":"2024-09-10T22:57:33.485308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487369+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:57:33.487369+0000","last_peered":"2024-09-10T22:57:33.487369+0000","last_clean":"2024-09-10T22:57:33.487369+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:57:33.487369+0000","last_undegraded":"2024-09-10T22:57:33.487369+0000","last_fullsized":"2024-09-10T22:57:33.487369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268610+0000","last_change":"2024-09-10T22:56:18.924551+0000","last_active":"2024-09-10T22:56:31.268610+0000","last_peered":"2024-09-10T22:56:31.268610+0000","last_clean":"2024-09-10T22:56:31.268610+0000","last_became_active":"2024-09-10T22:56:18.923067+0000","last_became_peered":"2024-09-10T22:56:18.923067+0000","last_unstale":"2024-09-10T22:56:31.268610+0000","last_undegraded":"2024-09-10T22:56:31.268610+0000","last_fullsized":"2024-09-10T22:56:31.268610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:55.514830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.267628+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:56:31.267628+0000","last_peered":"2024-09-10T22:56:31.267628+0000","last_clean":"2024-09-10T22:56:31.267628+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:56:31.267628+0000","last_undegraded":"2024-09-10T22:56:31.267628+0000","last_fullsized":"2024-09-10T22:56:31.267628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.114833+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:56:31.114833+0000","last_peered":"2024-09-10T22:56:31.114833+0000","last_clean":"2024-09-10T22:56:31.114833+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:56:31.114833+0000","last_undegraded":"2024-09-10T22:56:31.114833+0000","last_fullsized":"2024-09-10T22:56:31.114833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269828+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:56:31.269828+0000","last_peered":"2024-09-10T22:56:31.269828+0000","last_clean":"2024-09-10T22:56:31.269828+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:56:31.269828+0000","last_undegraded":"2024-09-10T22:56:31.269828+0000","last_fullsized":"2024-09-10T22:56:31.269828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269912+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:56:31.269912+0000","last_peered":"2024-09-10T22:56:31.269912+0000","last_clean":"2024-09-10T22:56:31.269912+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:56:31.269912+0000","last_undegraded":"2024-09-10T22:56:31.269912+0000","last_fullsized":"2024-09-10T22:56:31.269912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484370+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:57:32.484370+0000","last_peered":"2024-09-10T22:57:32.484370+0000","last_clean":"2024-09-10T22:57:32.484370+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:57:32.484370+0000","last_undegraded":"2024-09-10T22:57:32.484370+0000","last_fullsized":"2024-09-10T22:57:32.484370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56405,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":135,"num_read_kb":128,"num_write":162,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":178758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612638,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":479658,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":1119330,"data_compressed":53559,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":483990,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":479814,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":1085478,"data_compressed":56532,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":475070,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":1081066,"data_compressed":56532,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":479720,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":494545,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1024000,"data_stored":543145,"data_compressed":20675,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":541404,"data_compressed":20675,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":476458,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":57682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:57:36.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:57:36.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:36 smithi092 ceph-mon[34223]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:36.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:36 smithi067 ceph-mon[30939]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:36.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:36 smithi145 ceph-mon[34397]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:37.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 -- 172.21.15.67:0/4073563829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc104fc0 msgr2=0x7f66cc1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 --2- 172.21.15.67:0/4073563829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc104fc0 0x7f66cc1053a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009900 tx=0x7f66bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:57:37.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 -- 172.21.15.67:0/4073563829 shutdown_connections 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 --2- 172.21.15.67:0/4073563829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc10b350 0x7f66cc111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 --2- 172.21.15.67:0/4073563829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10a9d0 0x7f66cc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 --2- 172.21.15.67:0/4073563829 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc104fc0 0x7f66cc1053a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 -- 172.21.15.67:0/4073563829 >> 172.21.15.67:0/4073563829 conn(0x7f66cc1009f0 msgr2=0x7f66cc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 -- 172.21.15.67:0/4073563829 shutdown_connections 2024-09-10T22:57:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.405+0000 7f66d2a00640 1 -- 172.21.15.67:0/4073563829 wait complete. 2024-09-10T22:57:37.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.406+0000 7f66d2a00640 1 Processor -- start 2024-09-10T22:57:37.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.406+0000 7f66d2a00640 1 -- start start 2024-09-10T22:57:37.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc104fc0 0x7f66cc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10b350 0x7f66cc1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc114e10 con 0x7f66cc10a9d0 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc114c90 con 0x7f66cc104fc0 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d2a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc114f90 con 0x7f66cc10b350 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46254/0 (socket says 172.21.15.67:46254) 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 -- 172.21.15.67:0/357320 learned_addr learned my addr 172.21.15.67:0/357320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cbe00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc104fc0 0x7f66cc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66d0a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10b350 0x7f66cc1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 -- 172.21.15.67:0/357320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10b350 msgr2=0x7f66cc1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10b350 0x7f66cc1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 -- 172.21.15.67:0/357320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc104fc0 msgr2=0x7f66cc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.407+0000 7f66cb400640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc104fc0 0x7f66cc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66cb400640 1 -- 172.21.15.67:0/357320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc1a1470 con 0x7f66cc10a9d0 2024-09-10T22:57:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66cb400640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66b800d950 tx=0x7f66b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8017070 con 0x7f66cc10a9d0 2024-09-10T22:57:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66b8009c30 con 0x7f66cc10a9d0 2024-09-10T22:57:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b800f620 con 0x7f66cc10a9d0 2024-09-10T22:57:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc1a1760 con 0x7f66cc10a9d0 2024-09-10T22:57:37.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.408+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1a7e20 con 0x7f66cc10a9d0 2024-09-10T22:57:37.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.410+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66cc1053a0 con 0x7f66cc10a9d0 2024-09-10T22:57:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.413+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f66b8010460 con 0x7f66cc10a9d0 2024-09-10T22:57:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.414+0000 7f66c8c00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 0x7f66b00b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.414+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f66b80e5e00 con 0x7f66cc10a9d0 2024-09-10T22:57:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.415+0000 7f66cbe00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 0x7f66b00b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.415+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66b80ac760 con 0x7f66cc10a9d0 2024-09-10T22:57:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.415+0000 7f66cbe00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 0x7f66b00b7900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0057c0 tx=0x7f66bc0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:37 smithi067 ceph-mon[30939]: pgmap v347: 129 pgs: 1 activating, 2 peering, 126 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:37.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.610+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66cc063590 con 0x7f66b00b5440 2024-09-10T22:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.616+0000 7f66c8c00640 1 -- 172.21.15.67:0/357320 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340608 (secure 0 0 0) 0x7f66cc063590 con 0x7f66b00b5440 2024-09-10T22:57:37.621 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 msgr2=0x7f66b00b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 0x7f66b00b7900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0057c0 tx=0x7f66bc0058d0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 msgr2=0x7f66cc19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66b800d950 tx=0x7f66b800de10 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 shutdown_connections 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66b00b5440 0x7f66b00b7900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66cc10b350 0x7f66cc1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66cc10a9d0 0x7f66cc19b900 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 --2- 172.21.15.67:0/357320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66cc104fc0 0x7f66cc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 >> 172.21.15.67:0/357320 conn(0x7f66cc1009f0 msgr2=0x7f66cc101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 shutdown_connections 2024-09-10T22:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:37.621+0000 7f66d2a00640 1 -- 172.21.15.67:0/357320 wait complete. 2024-09-10T22:57:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:37 smithi092 ceph-mon[34223]: pgmap v347: 129 pgs: 1 activating, 2 peering, 126 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:37 smithi145 ceph-mon[34397]: pgmap v347: 129 pgs: 1 activating, 2 peering, 126 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s; 126 B/s, 0 objects/s recovering 2024-09-10T22:57:38.429 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T22:57:36.076007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651272,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":288,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2952853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356684,"kb_used_data":11544,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585140,"statfs":{"total":1151940427776,"available":1151575183360,"internally_reserved":0,"allocated":11821056,"data_stored":7830710,"data_compressed":364600,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13445,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":55,"num_read_kb":42,"num_write":11,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002288"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:10.425984+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:10.425984+0000","last_peered":"2024-09-10T22:57:10.425984+0000","last_clean":"2024-09-10T22:57:10.425984+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:10.425984+0000","last_undegraded":"2024-09-10T22:57:10.425984+0000","last_fullsized":"2024-09-10T22:57:10.425984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483317+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:57:33.483317+0000","last_peered":"2024-09-10T22:57:33.483317+0000","last_clean":"2024-09-10T22:57:33.483317+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:57:33.483317+0000","last_undegraded":"2024-09-10T22:57:33.483317+0000","last_fullsized":"2024-09-10T22:57:33.483317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476402+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:57:32.476402+0000","last_peered":"2024-09-10T22:57:32.476402+0000","last_clean":"2024-09-10T22:57:32.476402+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:57:32.476402+0000","last_undegraded":"2024-09-10T22:57:32.476402+0000","last_fullsized":"2024-09-10T22:57:32.476402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483508+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:57:33.483508+0000","last_peered":"2024-09-10T22:57:33.483508+0000","last_clean":"2024-09-10T22:57:33.483508+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:57:33.483508+0000","last_undegraded":"2024-09-10T22:57:33.483508+0000","last_fullsized":"2024-09-10T22:57:33.483508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492567+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:57:33.492567+0000","last_peered":"2024-09-10T22:57:33.492567+0000","last_clean":"2024-09-10T22:57:33.492567+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:57:33.492567+0000","last_undegraded":"2024-09-10T22:57:33.492567+0000","last_fullsized":"2024-09-10T22:57:33.492567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933763+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:57:33.933763+0000","last_peered":"2024-09-10T22:57:33.933763+0000","last_clean":"2024-09-10T22:57:33.933763+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:57:33.933763+0000","last_undegraded":"2024-09-10T22:57:33.933763+0000","last_fullsized":"2024-09-10T22:57:33.933763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495695+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:33.495695+0000","last_peered":"2024-09-10T22:57:33.495695+0000","last_clean":"2024-09-10T22:57:33.495695+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:33.495695+0000","last_undegraded":"2024-09-10T22:57:33.495695+0000","last_fullsized":"2024-09-10T22:57:33.495695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484689+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:57:33.484689+0000","last_peered":"2024-09-10T22:57:33.484689+0000","last_clean":"2024-09-10T22:57:33.484689+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:57:33.484689+0000","last_undegraded":"2024-09-10T22:57:33.484689+0000","last_fullsized":"2024-09-10T22:57:33.484689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933235+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:57:33.933235+0000","last_peered":"2024-09-10T22:57:33.933235+0000","last_clean":"2024-09-10T22:57:33.933235+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:57:33.933235+0000","last_undegraded":"2024-09-10T22:57:33.933235+0000","last_fullsized":"2024-09-10T22:57:33.933235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-10T22:56:19.912821+0000","last_change":"2024-09-10T22:57:32.903687+0000","last_active":"2024-09-10T22:57:32.903687+0000","last_peered":"2024-09-10T22:56:19.912821+0000","last_clean":"2024-09-10T22:56:19.912821+0000","last_became_active":"2024-09-10T22:56:16.890866+0000","last_became_peered":"2024-09-10T22:56:16.890866+0000","last_unstale":"2024-09-10T22:56:19.912821+0000","last_undegraded":"2024-09-10T22:56:19.912821+0000","last_fullsized":"2024-09-10T22:56:19.912821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"activating","last_fresh":"2024-09-10T22:57:33.485317+0000","last_change":"2024-09-10T22:57:33.485317+0000","last_active":"2024-09-10T22:57:32.482204+0000","last_peered":"2024-09-10T22:56:30.975209+0000","last_clean":"2024-09-10T22:56:30.975209+0000","last_became_active":"2024-09-10T22:56:18.921985+0000","last_became_peered":"2024-09-10T22:56:18.921985+0000","last_unstale":"2024-09-10T22:57:33.485317+0000","last_undegraded":"2024-09-10T22:57:33.485317+0000","last_fullsized":"2024-09-10T22:57:33.485317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":["5","1","4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933169+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:57:33.933169+0000","last_peered":"2024-09-10T22:57:33.933169+0000","last_clean":"2024-09-10T22:57:33.933169+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:57:33.933169+0000","last_undegraded":"2024-09-10T22:57:33.933169+0000","last_fullsized":"2024-09-10T22:57:33.933169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489400+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:57:33.489400+0000","last_peered":"2024-09-10T22:57:33.489400+0000","last_clean":"2024-09-10T22:57:33.489400+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:57:33.489400+0000","last_undegraded":"2024-09-10T22:57:33.489400+0000","last_fullsized":"2024-09-10T22:57:33.489400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494981+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:33.494981+0000","last_peered":"2024-09-10T22:57:33.494981+0000","last_clean":"2024-09-10T22:57:33.494981+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:33.494981+0000","last_undegraded":"2024-09-10T22:57:33.494981+0000","last_fullsized":"2024-09-10T22:57:33.494981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933680+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:57:33.933680+0000","last_peered":"2024-09-10T22:57:33.933680+0000","last_clean":"2024-09-10T22:57:33.933680+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:57:33.933680+0000","last_undegraded":"2024-09-10T22:57:33.933680+0000","last_fullsized":"2024-09-10T22:57:33.933680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483072+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:57:33.483072+0000","last_peered":"2024-09-10T22:57:33.483072+0000","last_clean":"2024-09-10T22:57:33.483072+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:57:33.483072+0000","last_undegraded":"2024-09-10T22:57:33.483072+0000","last_fullsized":"2024-09-10T22:57:33.483072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476456+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:57:32.476456+0000","last_peered":"2024-09-10T22:57:32.476456+0000","last_clean":"2024-09-10T22:57:32.476456+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:57:32.476456+0000","last_undegraded":"2024-09-10T22:57:32.476456+0000","last_fullsized":"2024-09-10T22:57:32.476456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933688+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:57:33.933688+0000","last_peered":"2024-09-10T22:57:33.933688+0000","last_clean":"2024-09-10T22:57:33.933688+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:57:33.933688+0000","last_undegraded":"2024-09-10T22:57:33.933688+0000","last_fullsized":"2024-09-10T22:57:33.933688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.483016+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:57:32.483016+0000","last_peered":"2024-09-10T22:57:32.483016+0000","last_clean":"2024-09-10T22:57:32.483016+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:57:32.483016+0000","last_undegraded":"2024-09-10T22:57:32.483016+0000","last_fullsized":"2024-09-10T22:57:32.483016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482999+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:33.482999+0000","last_peered":"2024-09-10T22:57:33.482999+0000","last_clean":"2024-09-10T22:57:33.482999+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:33.482999+0000","last_undegraded":"2024-09-10T22:57:33.482999+0000","last_fullsized":"2024-09-10T22:57:33.482999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269937+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:56:31.269937+0000","last_peered":"2024-09-10T22:56:31.269937+0000","last_clean":"2024-09-10T22:56:31.269937+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:56:31.269937+0000","last_undegraded":"2024-09-10T22:56:31.269937+0000","last_fullsized":"2024-09-10T22:56:31.269937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493982+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:57:33.493982+0000","last_peered":"2024-09-10T22:57:33.493982+0000","last_clean":"2024-09-10T22:57:33.493982+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:57:33.493982+0000","last_undegraded":"2024-09-10T22:57:33.493982+0000","last_fullsized":"2024-09-10T22:57:33.493982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484357+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:33.484357+0000","last_peered":"2024-09-10T22:57:33.484357+0000","last_clean":"2024-09-10T22:57:33.484357+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:33.484357+0000","last_undegraded":"2024-09-10T22:57:33.484357+0000","last_fullsized":"2024-09-10T22:57:33.484357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933699+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:57:33.933699+0000","last_peered":"2024-09-10T22:57:33.933699+0000","last_clean":"2024-09-10T22:57:33.933699+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:57:33.933699+0000","last_undegraded":"2024-09-10T22:57:33.933699+0000","last_fullsized":"2024-09-10T22:57:33.933699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492434+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:57:33.492434+0000","last_peered":"2024-09-10T22:57:33.492434+0000","last_clean":"2024-09-10T22:57:33.492434+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:57:33.492434+0000","last_undegraded":"2024-09-10T22:57:33.492434+0000","last_fullsized":"2024-09-10T22:57:33.492434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934281+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:33.934281+0000","last_peered":"2024-09-10T22:57:33.934281+0000","last_clean":"2024-09-10T22:57:33.934281+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:33.934281+0000","last_undegraded":"2024-09-10T22:57:33.934281+0000","last_fullsized":"2024-09-10T22:57:33.934281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477032+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:57:32.477032+0000","last_peered":"2024-09-10T22:57:32.477032+0000","last_clean":"2024-09-10T22:57:32.477032+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:57:32.477032+0000","last_undegraded":"2024-09-10T22:57:32.477032+0000","last_fullsized":"2024-09-10T22:57:32.477032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483447+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:57:33.483447+0000","last_peered":"2024-09-10T22:57:33.483447+0000","last_clean":"2024-09-10T22:57:33.483447+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:57:33.483447+0000","last_undegraded":"2024-09-10T22:57:33.483447+0000","last_fullsized":"2024-09-10T22:57:33.483447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:57:27.916037+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:27.916037+0000","last_peered":"2024-09-10T22:57:27.916037+0000","last_clean":"2024-09-10T22:57:27.916037+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:27.916037+0000","last_undegraded":"2024-09-10T22:57:27.916037+0000","last_fullsized":"2024-09-10T22:57:27.916037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482709+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:57:33.482709+0000","last_peered":"2024-09-10T22:57:33.482709+0000","last_clean":"2024-09-10T22:57:33.482709+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:57:33.482709+0000","last_undegraded":"2024-09-10T22:57:33.482709+0000","last_fullsized":"2024-09-10T22:57:33.482709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636052+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:57:33.636052+0000","last_peered":"2024-09-10T22:57:33.636052+0000","last_clean":"2024-09-10T22:57:33.636052+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:57:33.636052+0000","last_undegraded":"2024-09-10T22:57:33.636052+0000","last_fullsized":"2024-09-10T22:57:33.636052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495704+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:33.495704+0000","last_peered":"2024-09-10T22:57:33.495704+0000","last_clean":"2024-09-10T22:57:33.495704+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:33.495704+0000","last_undegraded":"2024-09-10T22:57:33.495704+0000","last_fullsized":"2024-09-10T22:57:33.495704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.482103+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:32.482103+0000","last_peered":"2024-09-10T22:57:32.482103+0000","last_clean":"2024-09-10T22:57:32.482103+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:32.482103+0000","last_undegraded":"2024-09-10T22:57:32.482103+0000","last_fullsized":"2024-09-10T22:57:32.482103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489113+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:57:33.489113+0000","last_peered":"2024-09-10T22:57:33.489113+0000","last_clean":"2024-09-10T22:57:33.489113+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:57:33.489113+0000","last_undegraded":"2024-09-10T22:57:33.489113+0000","last_fullsized":"2024-09-10T22:57:33.489113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269684+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:56:31.269684+0000","last_peered":"2024-09-10T22:56:31.269684+0000","last_clean":"2024-09-10T22:56:31.269684+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:56:31.269684+0000","last_undegraded":"2024-09-10T22:56:31.269684+0000","last_fullsized":"2024-09-10T22:56:31.269684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493448+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:57:33.493448+0000","last_peered":"2024-09-10T22:57:33.493448+0000","last_clean":"2024-09-10T22:57:33.493448+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:57:33.493448+0000","last_undegraded":"2024-09-10T22:57:33.493448+0000","last_fullsized":"2024-09-10T22:57:33.493448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490092+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:57:33.490092+0000","last_peered":"2024-09-10T22:57:33.490092+0000","last_clean":"2024-09-10T22:57:33.490092+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:57:33.490092+0000","last_undegraded":"2024-09-10T22:57:33.490092+0000","last_fullsized":"2024-09-10T22:57:33.490092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489599+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:57:33.489599+0000","last_peered":"2024-09-10T22:57:33.489599+0000","last_clean":"2024-09-10T22:57:33.489599+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:57:33.489599+0000","last_undegraded":"2024-09-10T22:57:33.489599+0000","last_fullsized":"2024-09-10T22:57:33.489599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635906+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:57:33.635906+0000","last_peered":"2024-09-10T22:57:33.635906+0000","last_clean":"2024-09-10T22:57:33.635906+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:57:33.635906+0000","last_undegraded":"2024-09-10T22:57:33.635906+0000","last_fullsized":"2024-09-10T22:57:33.635906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484247+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:57:33.484247+0000","last_peered":"2024-09-10T22:57:33.484247+0000","last_clean":"2024-09-10T22:57:33.484247+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:57:33.484247+0000","last_undegraded":"2024-09-10T22:57:33.484247+0000","last_fullsized":"2024-09-10T22:57:33.484247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635823+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:57:33.635823+0000","last_peered":"2024-09-10T22:57:33.635823+0000","last_clean":"2024-09-10T22:57:33.635823+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:57:33.635823+0000","last_undegraded":"2024-09-10T22:57:33.635823+0000","last_fullsized":"2024-09-10T22:57:33.635823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636145+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:57:33.636145+0000","last_peered":"2024-09-10T22:57:33.636145+0000","last_clean":"2024-09-10T22:57:33.636145+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:57:33.636145+0000","last_undegraded":"2024-09-10T22:57:33.636145+0000","last_fullsized":"2024-09-10T22:57:33.636145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635652+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:57:33.635652+0000","last_peered":"2024-09-10T22:57:33.635652+0000","last_clean":"2024-09-10T22:57:33.635652+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:57:33.635652+0000","last_undegraded":"2024-09-10T22:57:33.635652+0000","last_fullsized":"2024-09-10T22:57:33.635652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487480+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:33.487480+0000","last_peered":"2024-09-10T22:57:33.487480+0000","last_clean":"2024-09-10T22:57:33.487480+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:33.487480+0000","last_undegraded":"2024-09-10T22:57:33.487480+0000","last_fullsized":"2024-09-10T22:57:33.487480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484718+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:57:33.484718+0000","last_peered":"2024-09-10T22:57:33.484718+0000","last_clean":"2024-09-10T22:57:33.484718+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:57:33.484718+0000","last_undegraded":"2024-09-10T22:57:33.484718+0000","last_fullsized":"2024-09-10T22:57:33.484718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269073+0000","last_change":"2024-09-10T22:56:18.920310+0000","last_active":"2024-09-10T22:56:31.269073+0000","last_peered":"2024-09-10T22:56:31.269073+0000","last_clean":"2024-09-10T22:56:31.269073+0000","last_became_active":"2024-09-10T22:56:18.918860+0000","last_became_peered":"2024-09-10T22:56:18.918860+0000","last_unstale":"2024-09-10T22:56:31.269073+0000","last_undegraded":"2024-09-10T22:56:31.269073+0000","last_fullsized":"2024-09-10T22:56:31.269073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:33.506508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934139+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:33.934139+0000","last_peered":"2024-09-10T22:57:33.934139+0000","last_clean":"2024-09-10T22:57:33.934139+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:33.934139+0000","last_undegraded":"2024-09-10T22:57:33.934139+0000","last_fullsized":"2024-09-10T22:57:33.934139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487351+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:57:33.487351+0000","last_peered":"2024-09-10T22:57:33.487351+0000","last_clean":"2024-09-10T22:57:33.487351+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:57:33.487351+0000","last_undegraded":"2024-09-10T22:57:33.487351+0000","last_fullsized":"2024-09-10T22:57:33.487351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487434+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:57:33.487434+0000","last_peered":"2024-09-10T22:57:33.487434+0000","last_clean":"2024-09-10T22:57:33.487434+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:57:33.487434+0000","last_undegraded":"2024-09-10T22:57:33.487434+0000","last_fullsized":"2024-09-10T22:57:33.487434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487965+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:57:33.487965+0000","last_peered":"2024-09-10T22:57:33.487965+0000","last_clean":"2024-09-10T22:57:33.487965+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:57:33.487965+0000","last_undegraded":"2024-09-10T22:57:33.487965+0000","last_fullsized":"2024-09-10T22:57:33.487965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477275+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:32.477275+0000","last_peered":"2024-09-10T22:57:32.477275+0000","last_clean":"2024-09-10T22:57:32.477275+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:32.477275+0000","last_undegraded":"2024-09-10T22:57:32.477275+0000","last_fullsized":"2024-09-10T22:57:32.477275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484515+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:57:33.484515+0000","last_peered":"2024-09-10T22:57:33.484515+0000","last_clean":"2024-09-10T22:57:33.484515+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:57:33.484515+0000","last_undegraded":"2024-09-10T22:57:33.484515+0000","last_fullsized":"2024-09-10T22:57:33.484515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495905+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:57:33.495905+0000","last_peered":"2024-09-10T22:57:33.495905+0000","last_clean":"2024-09-10T22:57:33.495905+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:57:33.495905+0000","last_undegraded":"2024-09-10T22:57:33.495905+0000","last_fullsized":"2024-09-10T22:57:33.495905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485826+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:57:33.485826+0000","last_peered":"2024-09-10T22:57:33.485826+0000","last_clean":"2024-09-10T22:57:33.485826+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:57:33.485826+0000","last_undegraded":"2024-09-10T22:57:33.485826+0000","last_fullsized":"2024-09-10T22:57:33.485826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492168+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:57:33.492168+0000","last_peered":"2024-09-10T22:57:33.492168+0000","last_clean":"2024-09-10T22:57:33.492168+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:57:33.492168+0000","last_undegraded":"2024-09-10T22:57:33.492168+0000","last_fullsized":"2024-09-10T22:57:33.492168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487627+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:57:33.487627+0000","last_peered":"2024-09-10T22:57:33.487627+0000","last_clean":"2024-09-10T22:57:33.487627+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:57:33.487627+0000","last_undegraded":"2024-09-10T22:57:33.487627+0000","last_fullsized":"2024-09-10T22:57:33.487627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636233+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:57:33.636233+0000","last_peered":"2024-09-10T22:57:33.636233+0000","last_clean":"2024-09-10T22:57:33.636233+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:57:33.636233+0000","last_undegraded":"2024-09-10T22:57:33.636233+0000","last_fullsized":"2024-09-10T22:57:33.636233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483887+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:57:33.483887+0000","last_peered":"2024-09-10T22:57:33.483887+0000","last_clean":"2024-09-10T22:57:33.483887+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:57:33.483887+0000","last_undegraded":"2024-09-10T22:57:33.483887+0000","last_fullsized":"2024-09-10T22:57:33.483887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484195+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:57:32.484195+0000","last_peered":"2024-09-10T22:57:32.484195+0000","last_clean":"2024-09-10T22:57:32.484195+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:57:32.484195+0000","last_undegraded":"2024-09-10T22:57:32.484195+0000","last_fullsized":"2024-09-10T22:57:32.484195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495716+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:33.495716+0000","last_peered":"2024-09-10T22:57:33.495716+0000","last_clean":"2024-09-10T22:57:33.495716+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:33.495716+0000","last_undegraded":"2024-09-10T22:57:33.495716+0000","last_fullsized":"2024-09-10T22:57:33.495716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484068+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:57:33.484068+0000","last_peered":"2024-09-10T22:57:33.484068+0000","last_clean":"2024-09-10T22:57:33.484068+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:57:33.484068+0000","last_undegraded":"2024-09-10T22:57:33.484068+0000","last_fullsized":"2024-09-10T22:57:33.484068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490385+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:57:33.490385+0000","last_peered":"2024-09-10T22:57:33.490385+0000","last_clean":"2024-09-10T22:57:33.490385+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:57:33.490385+0000","last_undegraded":"2024-09-10T22:57:33.490385+0000","last_fullsized":"2024-09-10T22:57:33.490385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-10T22:57:09.413906+0000","last_change":"2024-09-10T22:57:32.903638+0000","last_active":"2024-09-10T22:57:32.903638+0000","last_peered":"2024-09-10T22:57:09.413906+0000","last_clean":"2024-09-10T22:57:09.413906+0000","last_became_active":"2024-09-10T22:56:17.605996+0000","last_became_peered":"2024-09-10T22:56:17.605996+0000","last_unstale":"2024-09-10T22:57:09.413906+0000","last_undegraded":"2024-09-10T22:57:09.413906+0000","last_fullsized":"2024-09-10T22:57:09.413906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636127+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:57:33.636127+0000","last_peered":"2024-09-10T22:57:33.636127+0000","last_clean":"2024-09-10T22:57:33.636127+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:57:33.636127+0000","last_undegraded":"2024-09-10T22:57:33.636127+0000","last_fullsized":"2024-09-10T22:57:33.636127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491949+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:57:33.491949+0000","last_peered":"2024-09-10T22:57:33.491949+0000","last_clean":"2024-09-10T22:57:33.491949+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:57:33.491949+0000","last_undegraded":"2024-09-10T22:57:33.491949+0000","last_fullsized":"2024-09-10T22:57:33.491949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933486+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:57:33.933486+0000","last_peered":"2024-09-10T22:57:33.933486+0000","last_clean":"2024-09-10T22:57:33.933486+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:57:33.933486+0000","last_undegraded":"2024-09-10T22:57:33.933486+0000","last_fullsized":"2024-09-10T22:57:33.933486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270217+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:56:31.270217+0000","last_peered":"2024-09-10T22:56:31.270217+0000","last_clean":"2024-09-10T22:56:31.270217+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:56:31.270217+0000","last_undegraded":"2024-09-10T22:56:31.270217+0000","last_fullsized":"2024-09-10T22:56:31.270217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487381+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:57:33.487381+0000","last_peered":"2024-09-10T22:57:33.487381+0000","last_clean":"2024-09-10T22:57:33.487381+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:57:33.487381+0000","last_undegraded":"2024-09-10T22:57:33.487381+0000","last_fullsized":"2024-09-10T22:57:33.487381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484551+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:33.484551+0000","last_peered":"2024-09-10T22:57:33.484551+0000","last_clean":"2024-09-10T22:57:33.484551+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:33.484551+0000","last_undegraded":"2024-09-10T22:57:33.484551+0000","last_fullsized":"2024-09-10T22:57:33.484551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484140+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:32.484140+0000","last_peered":"2024-09-10T22:57:32.484140+0000","last_clean":"2024-09-10T22:57:32.484140+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:32.484140+0000","last_undegraded":"2024-09-10T22:57:32.484140+0000","last_fullsized":"2024-09-10T22:57:32.484140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635806+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:57:33.635806+0000","last_peered":"2024-09-10T22:57:33.635806+0000","last_clean":"2024-09-10T22:57:33.635806+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:57:33.635806+0000","last_undegraded":"2024-09-10T22:57:33.635806+0000","last_fullsized":"2024-09-10T22:57:33.635806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476668+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:32.476668+0000","last_peered":"2024-09-10T22:57:32.476668+0000","last_clean":"2024-09-10T22:57:32.476668+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:32.476668+0000","last_undegraded":"2024-09-10T22:57:32.476668+0000","last_fullsized":"2024-09-10T22:57:32.476668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485286+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:57:33.485286+0000","last_peered":"2024-09-10T22:57:33.485286+0000","last_clean":"2024-09-10T22:57:33.485286+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:57:33.485286+0000","last_undegraded":"2024-09-10T22:57:33.485286+0000","last_fullsized":"2024-09-10T22:57:33.485286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495545+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:57:33.495545+0000","last_peered":"2024-09-10T22:57:33.495545+0000","last_clean":"2024-09-10T22:57:33.495545+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:57:33.495545+0000","last_undegraded":"2024-09-10T22:57:33.495545+0000","last_fullsized":"2024-09-10T22:57:33.495545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:35.495008+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:57:35.495008+0000","last_peered":"2024-09-10T22:57:35.495008+0000","last_clean":"2024-09-10T22:57:35.495008+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:57:35.495008+0000","last_undegraded":"2024-09-10T22:57:35.495008+0000","last_fullsized":"2024-09-10T22:57:35.495008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56669,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":82,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.269170+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:56:31.269170+0000","last_peered":"2024-09-10T22:56:31.269170+0000","last_clean":"2024-09-10T22:56:31.269170+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:56:31.269170+0000","last_undegraded":"2024-09-10T22:56:31.269170+0000","last_fullsized":"2024-09-10T22:56:31.269170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484785+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:57:33.484785+0000","last_peered":"2024-09-10T22:57:33.484785+0000","last_clean":"2024-09-10T22:57:33.484785+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:57:33.484785+0000","last_undegraded":"2024-09-10T22:57:33.484785+0000","last_fullsized":"2024-09-10T22:57:33.484785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485645+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:33.485645+0000","last_peered":"2024-09-10T22:57:33.485645+0000","last_clean":"2024-09-10T22:57:33.485645+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:33.485645+0000","last_undegraded":"2024-09-10T22:57:33.485645+0000","last_fullsized":"2024-09-10T22:57:33.485645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933615+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:57:33.933615+0000","last_peered":"2024-09-10T22:57:33.933615+0000","last_clean":"2024-09-10T22:57:33.933615+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:57:33.933615+0000","last_undegraded":"2024-09-10T22:57:33.933615+0000","last_fullsized":"2024-09-10T22:57:33.933615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492566+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:57:33.492566+0000","last_peered":"2024-09-10T22:57:33.492566+0000","last_clean":"2024-09-10T22:57:33.492566+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:57:33.492566+0000","last_undegraded":"2024-09-10T22:57:33.492566+0000","last_fullsized":"2024-09-10T22:57:33.492566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490487+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:33.490487+0000","last_peered":"2024-09-10T22:57:33.490487+0000","last_clean":"2024-09-10T22:57:33.490487+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:33.490487+0000","last_undegraded":"2024-09-10T22:57:33.490487+0000","last_fullsized":"2024-09-10T22:57:33.490487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268692+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:56:31.268692+0000","last_peered":"2024-09-10T22:56:31.268692+0000","last_clean":"2024-09-10T22:56:31.268692+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:56:31.268692+0000","last_undegraded":"2024-09-10T22:56:31.268692+0000","last_fullsized":"2024-09-10T22:56:31.268692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485175+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:57:33.485175+0000","last_peered":"2024-09-10T22:57:33.485175+0000","last_clean":"2024-09-10T22:57:33.485175+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:57:33.485175+0000","last_undegraded":"2024-09-10T22:57:33.485175+0000","last_fullsized":"2024-09-10T22:57:33.485175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:34.495769+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:57:34.495769+0000","last_peered":"2024-09-10T22:57:34.495769+0000","last_clean":"2024-09-10T22:57:34.495769+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:57:34.495769+0000","last_undegraded":"2024-09-10T22:57:34.495769+0000","last_fullsized":"2024-09-10T22:57:34.495769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482848+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:57:33.482848+0000","last_peered":"2024-09-10T22:57:33.482848+0000","last_clean":"2024-09-10T22:57:33.482848+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:57:33.482848+0000","last_undegraded":"2024-09-10T22:57:33.482848+0000","last_fullsized":"2024-09-10T22:57:33.482848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492182+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:57:33.492182+0000","last_peered":"2024-09-10T22:57:33.492182+0000","last_clean":"2024-09-10T22:57:33.492182+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:57:33.492182+0000","last_undegraded":"2024-09-10T22:57:33.492182+0000","last_fullsized":"2024-09-10T22:57:33.492182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.268563+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:56:31.268563+0000","last_peered":"2024-09-10T22:56:31.268563+0000","last_clean":"2024-09-10T22:56:31.268563+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:56:31.268563+0000","last_undegraded":"2024-09-10T22:56:31.268563+0000","last_fullsized":"2024-09-10T22:56:31.268563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484688+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:57:33.484688+0000","last_peered":"2024-09-10T22:57:33.484688+0000","last_clean":"2024-09-10T22:57:33.484688+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:57:33.484688+0000","last_undegraded":"2024-09-10T22:57:33.484688+0000","last_fullsized":"2024-09-10T22:57:33.484688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484099+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:32.484099+0000","last_peered":"2024-09-10T22:57:32.484099+0000","last_clean":"2024-09-10T22:57:32.484099+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:32.484099+0000","last_undegraded":"2024-09-10T22:57:32.484099+0000","last_fullsized":"2024-09-10T22:57:32.484099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635694+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:33.635694+0000","last_peered":"2024-09-10T22:57:33.635694+0000","last_clean":"2024-09-10T22:57:33.635694+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:33.635694+0000","last_undegraded":"2024-09-10T22:57:33.635694+0000","last_fullsized":"2024-09-10T22:57:33.635694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489621+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:33.489621+0000","last_peered":"2024-09-10T22:57:33.489621+0000","last_clean":"2024-09-10T22:57:33.489621+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:33.489621+0000","last_undegraded":"2024-09-10T22:57:33.489621+0000","last_fullsized":"2024-09-10T22:57:33.489621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484446+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:57:33.484446+0000","last_peered":"2024-09-10T22:57:33.484446+0000","last_clean":"2024-09-10T22:57:33.484446+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:57:33.484446+0000","last_undegraded":"2024-09-10T22:57:33.484446+0000","last_fullsized":"2024-09-10T22:57:33.484446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476645+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:57:32.476645+0000","last_peered":"2024-09-10T22:57:32.476645+0000","last_clean":"2024-09-10T22:57:32.476645+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:57:32.476645+0000","last_undegraded":"2024-09-10T22:57:32.476645+0000","last_fullsized":"2024-09-10T22:57:32.476645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.497103+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:57:33.497103+0000","last_peered":"2024-09-10T22:57:33.497103+0000","last_clean":"2024-09-10T22:57:33.497103+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:57:33.497103+0000","last_undegraded":"2024-09-10T22:57:33.497103+0000","last_fullsized":"2024-09-10T22:57:33.497103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494636+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:57:33.494636+0000","last_peered":"2024-09-10T22:57:33.494636+0000","last_clean":"2024-09-10T22:57:33.494636+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:57:33.494636+0000","last_undegraded":"2024-09-10T22:57:33.494636+0000","last_fullsized":"2024-09-10T22:57:33.494636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476809+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:57:32.476809+0000","last_peered":"2024-09-10T22:57:32.476809+0000","last_clean":"2024-09-10T22:57:32.476809+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:57:32.476809+0000","last_undegraded":"2024-09-10T22:57:32.476809+0000","last_fullsized":"2024-09-10T22:57:32.476809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.482461+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:57:32.482461+0000","last_peered":"2024-09-10T22:57:32.482461+0000","last_clean":"2024-09-10T22:57:32.482461+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:57:32.482461+0000","last_undegraded":"2024-09-10T22:57:32.482461+0000","last_fullsized":"2024-09-10T22:57:32.482461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477147+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:57:32.477147+0000","last_peered":"2024-09-10T22:57:32.477147+0000","last_clean":"2024-09-10T22:57:32.477147+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:57:32.477147+0000","last_undegraded":"2024-09-10T22:57:32.477147+0000","last_fullsized":"2024-09-10T22:57:32.477147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476565+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:57:32.476565+0000","last_peered":"2024-09-10T22:57:32.476565+0000","last_clean":"2024-09-10T22:57:32.476565+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:57:32.476565+0000","last_undegraded":"2024-09-10T22:57:32.476565+0000","last_fullsized":"2024-09-10T22:57:32.476565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492236+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:57:33.492236+0000","last_peered":"2024-09-10T22:57:33.492236+0000","last_clean":"2024-09-10T22:57:33.492236+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:57:33.492236+0000","last_undegraded":"2024-09-10T22:57:33.492236+0000","last_fullsized":"2024-09-10T22:57:33.492236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636049+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:57:33.636049+0000","last_peered":"2024-09-10T22:57:33.636049+0000","last_clean":"2024-09-10T22:57:33.636049+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:57:33.636049+0000","last_undegraded":"2024-09-10T22:57:33.636049+0000","last_fullsized":"2024-09-10T22:57:33.636049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.483994+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:57:32.483994+0000","last_peered":"2024-09-10T22:57:32.483994+0000","last_clean":"2024-09-10T22:57:32.483994+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:57:32.483994+0000","last_undegraded":"2024-09-10T22:57:32.483994+0000","last_fullsized":"2024-09-10T22:57:32.483994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.476977+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:57:32.476977+0000","last_peered":"2024-09-10T22:57:32.476977+0000","last_clean":"2024-09-10T22:57:32.476977+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:57:32.476977+0000","last_undegraded":"2024-09-10T22:57:32.476977+0000","last_fullsized":"2024-09-10T22:57:32.476977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483396+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:57:33.483396+0000","last_peered":"2024-09-10T22:57:33.483396+0000","last_clean":"2024-09-10T22:57:33.483396+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:57:33.483396+0000","last_undegraded":"2024-09-10T22:57:33.483396+0000","last_fullsized":"2024-09-10T22:57:33.483396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484099+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:57:33.484099+0000","last_peered":"2024-09-10T22:57:33.484099+0000","last_clean":"2024-09-10T22:57:33.484099+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:57:33.484099+0000","last_undegraded":"2024-09-10T22:57:33.484099+0000","last_fullsized":"2024-09-10T22:57:33.484099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482935+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:57:33.482935+0000","last_peered":"2024-09-10T22:57:33.482935+0000","last_clean":"2024-09-10T22:57:33.482935+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:57:33.482935+0000","last_undegraded":"2024-09-10T22:57:33.482935+0000","last_fullsized":"2024-09-10T22:57:33.482935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484667+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:57:33.484667+0000","last_peered":"2024-09-10T22:57:33.484667+0000","last_clean":"2024-09-10T22:57:33.484667+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:57:33.484667+0000","last_undegraded":"2024-09-10T22:57:33.484667+0000","last_fullsized":"2024-09-10T22:57:33.484667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635935+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:57:33.635935+0000","last_peered":"2024-09-10T22:57:33.635935+0000","last_clean":"2024-09-10T22:57:33.635935+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:57:33.635935+0000","last_undegraded":"2024-09-10T22:57:33.635935+0000","last_fullsized":"2024-09-10T22:57:33.635935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484014+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:57:32.484014+0000","last_peered":"2024-09-10T22:57:32.484014+0000","last_clean":"2024-09-10T22:57:32.484014+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:57:32.484014+0000","last_undegraded":"2024-09-10T22:57:32.484014+0000","last_fullsized":"2024-09-10T22:57:32.484014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.477049+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:57:32.477049+0000","last_peered":"2024-09-10T22:57:32.477049+0000","last_clean":"2024-09-10T22:57:32.477049+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:57:32.477049+0000","last_undegraded":"2024-09-10T22:57:32.477049+0000","last_fullsized":"2024-09-10T22:57:32.477049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T22:56:31.270668+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:56:31.270668+0000","last_peered":"2024-09-10T22:56:31.270668+0000","last_clean":"2024-09-10T22:56:31.270668+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:56:31.270668+0000","last_undegraded":"2024-09-10T22:56:31.270668+0000","last_fullsized":"2024-09-10T22:56:31.270668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484410+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:57:32.484410+0000","last_peered":"2024-09-10T22:57:32.484410+0000","last_clean":"2024-09-10T22:57:32.484410+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:57:32.484410+0000","last_undegraded":"2024-09-10T22:57:32.484410+0000","last_fullsized":"2024-09-10T22:57:32.484410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484224+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:57:33.484224+0000","last_peered":"2024-09-10T22:57:33.484224+0000","last_clean":"2024-09-10T22:57:33.484224+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:57:33.484224+0000","last_undegraded":"2024-09-10T22:57:33.484224+0000","last_fullsized":"2024-09-10T22:57:33.484224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491907+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:57:33.491907+0000","last_peered":"2024-09-10T22:57:33.491907+0000","last_clean":"2024-09-10T22:57:33.491907+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:57:33.491907+0000","last_undegraded":"2024-09-10T22:57:33.491907+0000","last_fullsized":"2024-09-10T22:57:33.491907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491892+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:57:33.491892+0000","last_peered":"2024-09-10T22:57:33.491892+0000","last_clean":"2024-09-10T22:57:33.491892+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:57:33.491892+0000","last_undegraded":"2024-09-10T22:57:33.491892+0000","last_fullsized":"2024-09-10T22:57:33.491892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635943+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:57:33.635943+0000","last_peered":"2024-09-10T22:57:33.635943+0000","last_clean":"2024-09-10T22:57:33.635943+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:57:33.635943+0000","last_undegraded":"2024-09-10T22:57:33.635943+0000","last_fullsized":"2024-09-10T22:57:33.635943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934287+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:57:33.934287+0000","last_peered":"2024-09-10T22:57:33.934287+0000","last_clean":"2024-09-10T22:57:33.934287+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:57:33.934287+0000","last_undegraded":"2024-09-10T22:57:33.934287+0000","last_fullsized":"2024-09-10T22:57:33.934287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487645+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:57:33.487645+0000","last_peered":"2024-09-10T22:57:33.487645+0000","last_clean":"2024-09-10T22:57:33.487645+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:57:33.487645+0000","last_undegraded":"2024-09-10T22:57:33.487645+0000","last_fullsized":"2024-09-10T22:57:33.487645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490348+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:57:33.490348+0000","last_peered":"2024-09-10T22:57:33.490348+0000","last_clean":"2024-09-10T22:57:33.490348+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:57:33.490348+0000","last_undegraded":"2024-09-10T22:57:33.490348+0000","last_fullsized":"2024-09-10T22:57:33.490348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485308+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:57:33.485308+0000","last_peered":"2024-09-10T22:57:33.485308+0000","last_clean":"2024-09-10T22:57:33.485308+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:57:33.485308+0000","last_undegraded":"2024-09-10T22:57:33.485308+0000","last_fullsized":"2024-09-10T22:57:33.485308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487369+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:57:33.487369+0000","last_peered":"2024-09-10T22:57:33.487369+0000","last_clean":"2024-09-10T22:57:33.487369+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:57:33.487369+0000","last_undegraded":"2024-09-10T22:57:33.487369+0000","last_fullsized":"2024-09-10T22:57:33.487369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495990+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:57:33.495990+0000","last_peered":"2024-09-10T22:57:33.495990+0000","last_clean":"2024-09-10T22:57:33.495990+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:57:33.495990+0000","last_undegraded":"2024-09-10T22:57:33.495990+0000","last_fullsized":"2024-09-10T22:57:33.495990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494336+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:57:33.494336+0000","last_peered":"2024-09-10T22:57:33.494336+0000","last_clean":"2024-09-10T22:57:33.494336+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:57:33.494336+0000","last_undegraded":"2024-09-10T22:57:33.494336+0000","last_fullsized":"2024-09-10T22:57:33.494336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933856+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:57:33.933856+0000","last_peered":"2024-09-10T22:57:33.933856+0000","last_clean":"2024-09-10T22:57:33.933856+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:57:33.933856+0000","last_undegraded":"2024-09-10T22:57:33.933856+0000","last_fullsized":"2024-09-10T22:57:33.933856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492277+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:57:33.492277+0000","last_peered":"2024-09-10T22:57:33.492277+0000","last_clean":"2024-09-10T22:57:33.492277+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:57:33.492277+0000","last_undegraded":"2024-09-10T22:57:33.492277+0000","last_fullsized":"2024-09-10T22:57:33.492277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492372+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:57:33.492372+0000","last_peered":"2024-09-10T22:57:33.492372+0000","last_clean":"2024-09-10T22:57:33.492372+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:57:33.492372+0000","last_undegraded":"2024-09-10T22:57:33.492372+0000","last_fullsized":"2024-09-10T22:57:33.492372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T22:57:32.484370+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:57:32.484370+0000","last_peered":"2024-09-10T22:57:32.484370+0000","last_clean":"2024-09-10T22:57:32.484370+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:57:32.484370+0000","last_undegraded":"2024-09-10T22:57:32.484370+0000","last_fullsized":"2024-09-10T22:57:32.484370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60670,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":159,"num_read_kb":147,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":183023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612638,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":479658,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":1142876,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":483990,"data_compressed":17702,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":498983,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":1085478,"data_compressed":56532,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":494351,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":1081066,"data_compressed":56532,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":489475,"data_compressed":20675,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":494545,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1024000,"data_stored":543145,"data_compressed":20675,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":541404,"data_compressed":20675,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705760,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":495739,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":57682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:57:38.434 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-10T22:57:38.434 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.1c', 'version': "79'2", 'reported_seq': 2, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-10T22:56:19.912821+0000', 'last_change': '2024-09-10T22:57:32.903687+0000', 'last_active': '2024-09-10T22:57:32.903687+0000', 'last_peered': '2024-09-10T22:56:19.912821+0000', 'last_clean': '2024-09-10T22:56:19.912821+0000', 'last_became_active': '2024-09-10T22:56:16.890866+0000', 'last_became_peered': '2024-09-10T22:56:16.890866+0000', 'last_unstale': '2024-09-10T22:56:19.912821+0000', 'last_undegraded': '2024-09-10T22:56:19.912821+0000', 'last_fullsized': '2024-09-10T22:56:19.912821+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 76, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'last_clean_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 1, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 1, 5], 'acting': [11, 1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-09-10T22:57:38.435 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T22:57:38.435 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 30, 'reported_epoch': 87, 'state': 'activating', 'last_fresh': '2024-09-10T22:57:33.485317+0000', 'last_change': '2024-09-10T22:57:33.485317+0000', 'last_active': '2024-09-10T22:57:32.482204+0000', 'last_peered': '2024-09-10T22:56:30.975209+0000', 'last_clean': '2024-09-10T22:56:30.975209+0000', 'last_became_active': '2024-09-10T22:56:18.921985+0000', 'last_became_peered': '2024-09-10T22:56:18.921985+0000', 'last_unstale': '2024-09-10T22:57:33.485317+0000', 'last_undegraded': '2024-09-10T22:57:33.485317+0000', 'last_fullsized': '2024-09-10T22:57:33.485317+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:56:17.875402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:56:17.875402+0000', 'last_clean_scrub_stamp': '2024-09-10T22:56:17.875402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6, 1], 'acting': [5, 6, 1], 'avail_no_missing': ['5', '1', '4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:57:38.435 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-10T22:57:38.435 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.10', 'version': "85'15", 'reported_seq': 2, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-10T22:57:09.413906+0000', 'last_change': '2024-09-10T22:57:32.903638+0000', 'last_active': '2024-09-10T22:57:32.903638+0000', 'last_peered': '2024-09-10T22:57:09.413906+0000', 'last_clean': '2024-09-10T22:57:09.413906+0000', 'last_became_active': '2024-09-10T22:56:17.605996+0000', 'last_became_peered': '2024-09-10T22:56:17.605996+0000', 'last_unstale': '2024-09-10T22:57:09.413906+0000', 'last_undegraded': '2024-09-10T22:57:09.413906+0000', 'last_fullsized': '2024-09-10T22:57:09.413906+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 76, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'last_clean_scrub_stamp': '2024-09-10T22:56:15.859533+0000', 'objects_scrubbed': 0, 'log_size': 15, 'log_dups_size': 0, 'ondisk_log_size': 15, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 112, 'num_objects': 8, 'num_object_clones': 0, 'num_object_copies': 24, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 8, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 6, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 0, 4], 'acting': [11, 0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-09-10T22:57:38.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-10T22:57:38.694 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:38 smithi067 ceph-mon[30939]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:38.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:38 smithi092 ceph-mon[34223]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:38.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:38 smithi145 ceph-mon[34397]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 -- 172.21.15.67:0/2967367926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075f40 msgr2=0x7f102810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 --2- 172.21.15.67:0/2967367926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075f40 0x7f102810ee10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1010009900 tx=0x7f101002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 -- 172.21.15.67:0/2967367926 shutdown_connections 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 --2- 172.21.15.67:0/2967367926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f102810f3e0 0x7f10281117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 --2- 172.21.15.67:0/2967367926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075f40 0x7f102810ee10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 --2- 172.21.15.67:0/2967367926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1028075540 0x7f1028075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 -- 172.21.15.67:0/2967367926 >> 172.21.15.67:0/2967367926 conn(0x7f10280fe830 msgr2=0x7f1028100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 -- 172.21.15.67:0/2967367926 shutdown_connections 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.464+0000 7f102de00640 1 -- 172.21.15.67:0/2967367926 wait complete. 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 Processor -- start 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 -- start start 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1028075f40 0x7f102819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102810f3e0 0x7f10281a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1028114e80 con 0x7f1028075540 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1028114d00 con 0x7f102810f3e0 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.465+0000 7f102de00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1028115000 con 0x7f1028075f40 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35990/0 (socket says 172.21.15.67:35990) 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 -- 172.21.15.67:0/3236687926 learned_addr learned my addr 172.21.15.67:0/3236687926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1027400640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102810f3e0 0x7f10281a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f101e000640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1028075f40 0x7f102819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 -- 172.21.15.67:0/3236687926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1028075f40 msgr2=0x7f102819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1028075f40 0x7f102819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 -- 172.21.15.67:0/3236687926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102810f3e0 msgr2=0x7f10281a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102810f3e0 0x7f10281a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 -- 172.21.15.67:0/3236687926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281a14e0 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.466+0000 7f1026a00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f101800e9f0 tx=0x7f101800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.467+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800cd80 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.467+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10180108c0 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.467+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281a17d0 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.467+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180187c0 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.467+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281a8060 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.468+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1028076a50 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.470+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1018010430 con 0x7f1028075540 2024-09-10T22:57:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.471+0000 7f102ca00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 0x7f100c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:39.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.472+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f10180e6570 con 0x7f1028075540 2024-09-10T22:57:39.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.472+0000 7f101e000640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 0x7f100c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:39.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.473+0000 7f101e000640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 0x7f100c0b7900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1010006010 tx=0x7f1010005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:39.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.475+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10180aceb0 con 0x7f1028075540 2024-09-10T22:57:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.745+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1028075920 con 0x7f1028075540 2024-09-10T22:57:39.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:39 smithi067 ceph-mon[30939]: pgmap v348: 129 pgs: 1 activating, 128 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:39.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:39 smithi067 ceph-mon[30939]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:57:39.748 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:39 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T22:57:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.746+0000 7f102ca00640 1 -- 172.21.15.67:0/3236687926 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2479 (secure 0 0 0) 0x7f1028075920 con 0x7f1028075540 2024-09-10T22:57:39.748 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:39.748 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":561,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":87,"num_osds":12,"num_up_osds":12,"osd_up_since":1726008917,"num_in_osds":12,"osd_in_since":1726008890,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"activating","count":1}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":651272,"bytes_used":365318144,"bytes_avail":1151575109632,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":140,"recovering_keys_per_sec":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"read_bytes_sec":4734,"write_bytes_sec":2175,"read_op_per_sec":4,"write_op_per_sec":1},"fsmap":{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24664},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24676},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24691},{"filesystem_id":1,"rank":2,"name":"k","status":"up:active","gid":34631},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":34691},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":34730}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"services":{"dashboard":"https://172.21.15.67:8443/"}},"servicemap":{"epoch":29,"modified":"2024-09-10T22:57:24.072888+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 msgr2=0x7f100c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 0x7f100c0b7900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1010006010 tx=0x7f1010005a20 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 msgr2=0x7f102819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f101800e9f0 tx=0x7f101800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 shutdown_connections 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f100c0b5440 0x7f100c0b7900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f102810f3e0 0x7f10281a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1028075f40 0x7f102819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 --2- 172.21.15.67:0/3236687926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1028075540 0x7f102819b430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 >> 172.21.15.67:0/3236687926 conn(0x7f10280fe830 msgr2=0x7f10280ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 shutdown_connections 2024-09-10T22:57:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:39.749+0000 7f102de00640 1 -- 172.21.15.67:0/3236687926 wait complete. 2024-09-10T22:57:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:39 smithi092 ceph-mon[34223]: pgmap v348: 129 pgs: 1 activating, 128 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:39 smithi092 ceph-mon[34223]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:57:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:39 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T22:57:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:39 smithi145 ceph-mon[34397]: pgmap v348: 129 pgs: 1 activating, 128 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:39 smithi145 ceph-mon[34397]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:57:39.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:39 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T22:57:40.622 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-10T22:57:40.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:57:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:40 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3236687926' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-10T22:57:40.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:40 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3236687926' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-10T22:57:40.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:40 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3236687926' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-10T22:57:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 -- 172.21.15.67:0/2710342708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc102da0 msgr2=0x7f72bc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 --2- 172.21.15.67:0/2710342708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc102da0 0x7f72bc103180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 -- 172.21.15.67:0/2710342708 shutdown_connections 2024-09-10T22:57:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 --2- 172.21.15.67:0/2710342708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 0x7f72bc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 --2- 172.21.15.67:0/2710342708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc1087b0 0x7f72bc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 --2- 172.21.15.67:0/2710342708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc102da0 0x7f72bc103180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 -- 172.21.15.67:0/2710342708 >> 172.21.15.67:0/2710342708 conn(0x7f72bc0fe810 msgr2=0x7f72bc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 -- 172.21.15.67:0/2710342708 shutdown_connections 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.574+0000 7f72c2c00640 1 -- 172.21.15.67:0/2710342708 wait complete. 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.575+0000 7f72c2c00640 1 Processor -- start 2024-09-10T22:57:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.575+0000 7f72c2c00640 1 -- start start 2024-09-10T22:57:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc102da0 0x7f72bc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 0x7f72bc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112cf0 con 0x7f72bc1087b0 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112b70 con 0x7f72bc109130 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c2c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112e70 con 0x7f72bc102da0 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36000/0 (socket says 172.21.15.67:36000) 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72b3e00640 1 -- 172.21.15.67:0/1248215518 learned_addr learned my addr 172.21.15.67:0/1248215518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72c0c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 0x7f72bc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:41.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.576+0000 7f72bbe00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc102da0 0x7f72bc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 -- 172.21.15.67:0/1248215518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc102da0 msgr2=0x7f72bc199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc102da0 0x7f72bc199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 -- 172.21.15.67:0/1248215518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 msgr2=0x7f72bc19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 0x7f72bc19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 -- 172.21.15.67:0/1248215518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc19f2b0 con 0x7f72bc1087b0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b3e00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b4a0 tx=0x7f72a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4014070 con 0x7f72bc1087b0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc19f5a0 con 0x7f72bc1087b0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a4010aa0 con 0x7f72bc1087b0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4017b70 con 0x7f72bc1087b0 2024-09-10T22:57:41.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.577+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1a5ce0 con 0x7f72bc1087b0 2024-09-10T22:57:41.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.579+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc103180 con 0x7f72bc1087b0 2024-09-10T22:57:41.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.581+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f72a4010610 con 0x7f72bc1087b0 2024-09-10T22:57:41.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.583+0000 7f72b9600640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 0x7f72a00b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:41.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.583+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f72a40e56a0 con 0x7f72bc1087b0 2024-09-10T22:57:41.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.583+0000 7f72bbe00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 0x7f72a00b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:41.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.584+0000 7f72bbe00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 0x7f72a00b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72ac006010 tx=0x7f72ac005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:41.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.586+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f72a4010c50 con 0x7f72bc1087b0 2024-09-10T22:57:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:41 smithi092 ceph-mon[34223]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 5 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.779+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72bc063590 con 0x7f72a00b5440 2024-09-10T22:57:41.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:41 smithi067 ceph-mon[30939]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 5 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:41.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.785+0000 7f72b9600640 1 -- 172.21.15.67:0/1248215518 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340625 (secure 0 0 0) 0x7f72bc063590 con 0x7f72a00b5440 2024-09-10T22:57:41.791 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:41.793 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 msgr2=0x7f72a00b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 0x7f72a00b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72ac006010 tx=0x7f72ac005e80 comp rx=0 tx=0).stop 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 msgr2=0x7f72bc199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b4a0 tx=0x7f72a400b960 comp rx=0 tx=0).stop 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 shutdown_connections 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f72a00b5440 0x7f72a00b7900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc109130 0x7f72bc19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72bc1087b0 0x7f72bc199740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 --2- 172.21.15.67:0/1248215518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72bc102da0 0x7f72bc199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:41.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 >> 172.21.15.67:0/1248215518 conn(0x7f72bc0fe810 msgr2=0x7f72bc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:41.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 shutdown_connections 2024-09-10T22:57:41.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:41.789+0000 7f72c2c00640 1 -- 172.21.15.67:0/1248215518 wait complete. 2024-09-10T22:57:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:41 smithi145 ceph-mon[34397]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 5 op/s; 140 B/s, 1 objects/s recovering 2024-09-10T22:57:42.671 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T22:57:40.077152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651272,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":255,"num_read_kb":221,"num_write":290,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356820,"kb_used_data":11668,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585004,"statfs":{"total":1151940427776,"available":1151575044096,"internally_reserved":0,"allocated":11948032,"data_stored":7927820,"data_compressed":391371,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4265,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":23,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002259"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485580+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:33.485580+0000","last_peered":"2024-09-10T22:57:33.485580+0000","last_clean":"2024-09-10T22:57:33.485580+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:33.485580+0000","last_undegraded":"2024-09-10T22:57:33.485580+0000","last_fullsized":"2024-09-10T22:57:33.485580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483317+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:57:33.483317+0000","last_peered":"2024-09-10T22:57:33.483317+0000","last_clean":"2024-09-10T22:57:33.483317+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:57:33.483317+0000","last_undegraded":"2024-09-10T22:57:33.483317+0000","last_fullsized":"2024-09-10T22:57:33.483317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488147+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:57:33.488147+0000","last_peered":"2024-09-10T22:57:33.488147+0000","last_clean":"2024-09-10T22:57:33.488147+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:57:33.488147+0000","last_undegraded":"2024-09-10T22:57:33.488147+0000","last_fullsized":"2024-09-10T22:57:33.488147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483508+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:57:33.483508+0000","last_peered":"2024-09-10T22:57:33.483508+0000","last_clean":"2024-09-10T22:57:33.483508+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:57:33.483508+0000","last_undegraded":"2024-09-10T22:57:33.483508+0000","last_fullsized":"2024-09-10T22:57:33.483508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492567+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:57:33.492567+0000","last_peered":"2024-09-10T22:57:33.492567+0000","last_clean":"2024-09-10T22:57:33.492567+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:57:33.492567+0000","last_undegraded":"2024-09-10T22:57:33.492567+0000","last_fullsized":"2024-09-10T22:57:33.492567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933763+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:57:33.933763+0000","last_peered":"2024-09-10T22:57:33.933763+0000","last_clean":"2024-09-10T22:57:33.933763+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:57:33.933763+0000","last_undegraded":"2024-09-10T22:57:33.933763+0000","last_fullsized":"2024-09-10T22:57:33.933763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495695+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:33.495695+0000","last_peered":"2024-09-10T22:57:33.495695+0000","last_clean":"2024-09-10T22:57:33.495695+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:33.495695+0000","last_undegraded":"2024-09-10T22:57:33.495695+0000","last_fullsized":"2024-09-10T22:57:33.495695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484689+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:57:33.484689+0000","last_peered":"2024-09-10T22:57:33.484689+0000","last_clean":"2024-09-10T22:57:33.484689+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:57:33.484689+0000","last_undegraded":"2024-09-10T22:57:33.484689+0000","last_fullsized":"2024-09-10T22:57:33.484689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933235+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:57:33.933235+0000","last_peered":"2024-09-10T22:57:33.933235+0000","last_clean":"2024-09-10T22:57:33.933235+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:57:33.933235+0000","last_undegraded":"2024-09-10T22:57:33.933235+0000","last_fullsized":"2024-09-10T22:57:33.933235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.499755+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:57:33.499755+0000","last_peered":"2024-09-10T22:57:33.499755+0000","last_clean":"2024-09-10T22:57:33.499755+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:57:33.499755+0000","last_undegraded":"2024-09-10T22:57:33.499755+0000","last_fullsized":"2024-09-10T22:57:33.499755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.637598+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:57:33.637598+0000","last_peered":"2024-09-10T22:57:33.637598+0000","last_clean":"2024-09-10T22:57:33.637598+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:57:33.637598+0000","last_undegraded":"2024-09-10T22:57:33.637598+0000","last_fullsized":"2024-09-10T22:57:33.637598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933169+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:57:33.933169+0000","last_peered":"2024-09-10T22:57:33.933169+0000","last_clean":"2024-09-10T22:57:33.933169+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:57:33.933169+0000","last_undegraded":"2024-09-10T22:57:33.933169+0000","last_fullsized":"2024-09-10T22:57:33.933169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489400+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:57:33.489400+0000","last_peered":"2024-09-10T22:57:33.489400+0000","last_clean":"2024-09-10T22:57:33.489400+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:57:33.489400+0000","last_undegraded":"2024-09-10T22:57:33.489400+0000","last_fullsized":"2024-09-10T22:57:33.489400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494981+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:33.494981+0000","last_peered":"2024-09-10T22:57:33.494981+0000","last_clean":"2024-09-10T22:57:33.494981+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:33.494981+0000","last_undegraded":"2024-09-10T22:57:33.494981+0000","last_fullsized":"2024-09-10T22:57:33.494981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933680+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:57:33.933680+0000","last_peered":"2024-09-10T22:57:33.933680+0000","last_clean":"2024-09-10T22:57:33.933680+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:57:33.933680+0000","last_undegraded":"2024-09-10T22:57:33.933680+0000","last_fullsized":"2024-09-10T22:57:33.933680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483072+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:57:33.483072+0000","last_peered":"2024-09-10T22:57:33.483072+0000","last_clean":"2024-09-10T22:57:33.483072+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:57:33.483072+0000","last_undegraded":"2024-09-10T22:57:33.483072+0000","last_fullsized":"2024-09-10T22:57:33.483072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488646+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:57:33.488646+0000","last_peered":"2024-09-10T22:57:33.488646+0000","last_clean":"2024-09-10T22:57:33.488646+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:57:33.488646+0000","last_undegraded":"2024-09-10T22:57:33.488646+0000","last_fullsized":"2024-09-10T22:57:33.488646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933688+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:57:33.933688+0000","last_peered":"2024-09-10T22:57:33.933688+0000","last_clean":"2024-09-10T22:57:33.933688+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:57:33.933688+0000","last_undegraded":"2024-09-10T22:57:33.933688+0000","last_fullsized":"2024-09-10T22:57:33.933688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489595+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:57:33.489595+0000","last_peered":"2024-09-10T22:57:33.489595+0000","last_clean":"2024-09-10T22:57:33.489595+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:57:33.489595+0000","last_undegraded":"2024-09-10T22:57:33.489595+0000","last_fullsized":"2024-09-10T22:57:33.489595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482999+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:33.482999+0000","last_peered":"2024-09-10T22:57:33.482999+0000","last_clean":"2024-09-10T22:57:33.482999+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:33.482999+0000","last_undegraded":"2024-09-10T22:57:33.482999+0000","last_fullsized":"2024-09-10T22:57:33.482999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485365+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:57:33.485365+0000","last_peered":"2024-09-10T22:57:33.485365+0000","last_clean":"2024-09-10T22:57:33.485365+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:57:33.485365+0000","last_undegraded":"2024-09-10T22:57:33.485365+0000","last_fullsized":"2024-09-10T22:57:33.485365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493982+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:57:33.493982+0000","last_peered":"2024-09-10T22:57:33.493982+0000","last_clean":"2024-09-10T22:57:33.493982+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:57:33.493982+0000","last_undegraded":"2024-09-10T22:57:33.493982+0000","last_fullsized":"2024-09-10T22:57:33.493982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484357+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:33.484357+0000","last_peered":"2024-09-10T22:57:33.484357+0000","last_clean":"2024-09-10T22:57:33.484357+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:33.484357+0000","last_undegraded":"2024-09-10T22:57:33.484357+0000","last_fullsized":"2024-09-10T22:57:33.484357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933699+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:57:33.933699+0000","last_peered":"2024-09-10T22:57:33.933699+0000","last_clean":"2024-09-10T22:57:33.933699+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:57:33.933699+0000","last_undegraded":"2024-09-10T22:57:33.933699+0000","last_fullsized":"2024-09-10T22:57:33.933699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492434+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:57:33.492434+0000","last_peered":"2024-09-10T22:57:33.492434+0000","last_clean":"2024-09-10T22:57:33.492434+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:57:33.492434+0000","last_undegraded":"2024-09-10T22:57:33.492434+0000","last_fullsized":"2024-09-10T22:57:33.492434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934281+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:33.934281+0000","last_peered":"2024-09-10T22:57:33.934281+0000","last_clean":"2024-09-10T22:57:33.934281+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:33.934281+0000","last_undegraded":"2024-09-10T22:57:33.934281+0000","last_fullsized":"2024-09-10T22:57:33.934281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488401+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:57:33.488401+0000","last_peered":"2024-09-10T22:57:33.488401+0000","last_clean":"2024-09-10T22:57:33.488401+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:57:33.488401+0000","last_undegraded":"2024-09-10T22:57:33.488401+0000","last_fullsized":"2024-09-10T22:57:33.488401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483447+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:57:33.483447+0000","last_peered":"2024-09-10T22:57:33.483447+0000","last_clean":"2024-09-10T22:57:33.483447+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:57:33.483447+0000","last_undegraded":"2024-09-10T22:57:33.483447+0000","last_fullsized":"2024-09-10T22:57:33.483447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485392+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:33.485392+0000","last_peered":"2024-09-10T22:57:33.485392+0000","last_clean":"2024-09-10T22:57:33.485392+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:33.485392+0000","last_undegraded":"2024-09-10T22:57:33.485392+0000","last_fullsized":"2024-09-10T22:57:33.485392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482709+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:57:33.482709+0000","last_peered":"2024-09-10T22:57:33.482709+0000","last_clean":"2024-09-10T22:57:33.482709+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:57:33.482709+0000","last_undegraded":"2024-09-10T22:57:33.482709+0000","last_fullsized":"2024-09-10T22:57:33.482709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636052+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:57:33.636052+0000","last_peered":"2024-09-10T22:57:33.636052+0000","last_clean":"2024-09-10T22:57:33.636052+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:57:33.636052+0000","last_undegraded":"2024-09-10T22:57:33.636052+0000","last_fullsized":"2024-09-10T22:57:33.636052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495704+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:33.495704+0000","last_peered":"2024-09-10T22:57:33.495704+0000","last_clean":"2024-09-10T22:57:33.495704+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:33.495704+0000","last_undegraded":"2024-09-10T22:57:33.495704+0000","last_fullsized":"2024-09-10T22:57:33.495704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487731+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:33.487731+0000","last_peered":"2024-09-10T22:57:33.487731+0000","last_clean":"2024-09-10T22:57:33.487731+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:33.487731+0000","last_undegraded":"2024-09-10T22:57:33.487731+0000","last_fullsized":"2024-09-10T22:57:33.487731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489113+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:57:33.489113+0000","last_peered":"2024-09-10T22:57:33.489113+0000","last_clean":"2024-09-10T22:57:33.489113+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:57:33.489113+0000","last_undegraded":"2024-09-10T22:57:33.489113+0000","last_fullsized":"2024-09-10T22:57:33.489113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485163+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:57:33.485163+0000","last_peered":"2024-09-10T22:57:33.485163+0000","last_clean":"2024-09-10T22:57:33.485163+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:57:33.485163+0000","last_undegraded":"2024-09-10T22:57:33.485163+0000","last_fullsized":"2024-09-10T22:57:33.485163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493448+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:57:33.493448+0000","last_peered":"2024-09-10T22:57:33.493448+0000","last_clean":"2024-09-10T22:57:33.493448+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:57:33.493448+0000","last_undegraded":"2024-09-10T22:57:33.493448+0000","last_fullsized":"2024-09-10T22:57:33.493448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490092+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:57:33.490092+0000","last_peered":"2024-09-10T22:57:33.490092+0000","last_clean":"2024-09-10T22:57:33.490092+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:57:33.490092+0000","last_undegraded":"2024-09-10T22:57:33.490092+0000","last_fullsized":"2024-09-10T22:57:33.490092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489599+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:57:33.489599+0000","last_peered":"2024-09-10T22:57:33.489599+0000","last_clean":"2024-09-10T22:57:33.489599+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:57:33.489599+0000","last_undegraded":"2024-09-10T22:57:33.489599+0000","last_fullsized":"2024-09-10T22:57:33.489599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635906+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:57:33.635906+0000","last_peered":"2024-09-10T22:57:33.635906+0000","last_clean":"2024-09-10T22:57:33.635906+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:57:33.635906+0000","last_undegraded":"2024-09-10T22:57:33.635906+0000","last_fullsized":"2024-09-10T22:57:33.635906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484247+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:57:33.484247+0000","last_peered":"2024-09-10T22:57:33.484247+0000","last_clean":"2024-09-10T22:57:33.484247+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:57:33.484247+0000","last_undegraded":"2024-09-10T22:57:33.484247+0000","last_fullsized":"2024-09-10T22:57:33.484247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:38.484543+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:57:38.484543+0000","last_peered":"2024-09-10T22:57:38.484543+0000","last_clean":"2024-09-10T22:57:38.484543+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:57:38.484543+0000","last_undegraded":"2024-09-10T22:57:38.484543+0000","last_fullsized":"2024-09-10T22:57:38.484543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636145+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:57:33.636145+0000","last_peered":"2024-09-10T22:57:33.636145+0000","last_clean":"2024-09-10T22:57:33.636145+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:57:33.636145+0000","last_undegraded":"2024-09-10T22:57:33.636145+0000","last_fullsized":"2024-09-10T22:57:33.636145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635652+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:57:33.635652+0000","last_peered":"2024-09-10T22:57:33.635652+0000","last_clean":"2024-09-10T22:57:33.635652+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:57:33.635652+0000","last_undegraded":"2024-09-10T22:57:33.635652+0000","last_fullsized":"2024-09-10T22:57:33.635652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487480+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:33.487480+0000","last_peered":"2024-09-10T22:57:33.487480+0000","last_clean":"2024-09-10T22:57:33.487480+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:33.487480+0000","last_undegraded":"2024-09-10T22:57:33.487480+0000","last_fullsized":"2024-09-10T22:57:33.487480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484718+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:57:33.484718+0000","last_peered":"2024-09-10T22:57:33.484718+0000","last_clean":"2024-09-10T22:57:33.484718+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:57:33.484718+0000","last_undegraded":"2024-09-10T22:57:33.484718+0000","last_fullsized":"2024-09-10T22:57:33.484718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489346+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:57:33.489346+0000","last_peered":"2024-09-10T22:57:33.489346+0000","last_clean":"2024-09-10T22:57:33.489346+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:57:33.489346+0000","last_undegraded":"2024-09-10T22:57:33.489346+0000","last_fullsized":"2024-09-10T22:57:33.489346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934139+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:33.934139+0000","last_peered":"2024-09-10T22:57:33.934139+0000","last_clean":"2024-09-10T22:57:33.934139+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:33.934139+0000","last_undegraded":"2024-09-10T22:57:33.934139+0000","last_fullsized":"2024-09-10T22:57:33.934139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487351+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:57:33.487351+0000","last_peered":"2024-09-10T22:57:33.487351+0000","last_clean":"2024-09-10T22:57:33.487351+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:57:33.487351+0000","last_undegraded":"2024-09-10T22:57:33.487351+0000","last_fullsized":"2024-09-10T22:57:33.487351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487434+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:57:33.487434+0000","last_peered":"2024-09-10T22:57:33.487434+0000","last_clean":"2024-09-10T22:57:33.487434+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:57:33.487434+0000","last_undegraded":"2024-09-10T22:57:33.487434+0000","last_fullsized":"2024-09-10T22:57:33.487434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487965+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:57:33.487965+0000","last_peered":"2024-09-10T22:57:33.487965+0000","last_clean":"2024-09-10T22:57:33.487965+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:57:33.487965+0000","last_undegraded":"2024-09-10T22:57:33.487965+0000","last_fullsized":"2024-09-10T22:57:33.487965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489704+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:33.489704+0000","last_peered":"2024-09-10T22:57:33.489704+0000","last_clean":"2024-09-10T22:57:33.489704+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:33.489704+0000","last_undegraded":"2024-09-10T22:57:33.489704+0000","last_fullsized":"2024-09-10T22:57:33.489704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484515+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:57:33.484515+0000","last_peered":"2024-09-10T22:57:33.484515+0000","last_clean":"2024-09-10T22:57:33.484515+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:57:33.484515+0000","last_undegraded":"2024-09-10T22:57:33.484515+0000","last_fullsized":"2024-09-10T22:57:33.484515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495905+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:57:33.495905+0000","last_peered":"2024-09-10T22:57:33.495905+0000","last_clean":"2024-09-10T22:57:33.495905+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:57:33.495905+0000","last_undegraded":"2024-09-10T22:57:33.495905+0000","last_fullsized":"2024-09-10T22:57:33.495905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485826+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:57:33.485826+0000","last_peered":"2024-09-10T22:57:33.485826+0000","last_clean":"2024-09-10T22:57:33.485826+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:57:33.485826+0000","last_undegraded":"2024-09-10T22:57:33.485826+0000","last_fullsized":"2024-09-10T22:57:33.485826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492168+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:57:33.492168+0000","last_peered":"2024-09-10T22:57:33.492168+0000","last_clean":"2024-09-10T22:57:33.492168+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:57:33.492168+0000","last_undegraded":"2024-09-10T22:57:33.492168+0000","last_fullsized":"2024-09-10T22:57:33.492168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487627+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:57:33.487627+0000","last_peered":"2024-09-10T22:57:33.487627+0000","last_clean":"2024-09-10T22:57:33.487627+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:57:33.487627+0000","last_undegraded":"2024-09-10T22:57:33.487627+0000","last_fullsized":"2024-09-10T22:57:33.487627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636233+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:57:33.636233+0000","last_peered":"2024-09-10T22:57:33.636233+0000","last_clean":"2024-09-10T22:57:33.636233+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:57:33.636233+0000","last_undegraded":"2024-09-10T22:57:33.636233+0000","last_fullsized":"2024-09-10T22:57:33.636233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483887+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:57:33.483887+0000","last_peered":"2024-09-10T22:57:33.483887+0000","last_clean":"2024-09-10T22:57:33.483887+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:57:33.483887+0000","last_undegraded":"2024-09-10T22:57:33.483887+0000","last_fullsized":"2024-09-10T22:57:33.483887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489318+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:57:33.489318+0000","last_peered":"2024-09-10T22:57:33.489318+0000","last_clean":"2024-09-10T22:57:33.489318+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:57:33.489318+0000","last_undegraded":"2024-09-10T22:57:33.489318+0000","last_fullsized":"2024-09-10T22:57:33.489318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495716+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:33.495716+0000","last_peered":"2024-09-10T22:57:33.495716+0000","last_clean":"2024-09-10T22:57:33.495716+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:33.495716+0000","last_undegraded":"2024-09-10T22:57:33.495716+0000","last_fullsized":"2024-09-10T22:57:33.495716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484068+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:57:33.484068+0000","last_peered":"2024-09-10T22:57:33.484068+0000","last_clean":"2024-09-10T22:57:33.484068+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:57:33.484068+0000","last_undegraded":"2024-09-10T22:57:33.484068+0000","last_fullsized":"2024-09-10T22:57:33.484068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490385+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:57:33.490385+0000","last_peered":"2024-09-10T22:57:33.490385+0000","last_clean":"2024-09-10T22:57:33.490385+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:57:33.490385+0000","last_undegraded":"2024-09-10T22:57:33.490385+0000","last_fullsized":"2024-09-10T22:57:33.490385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.968470+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:57:33.968470+0000","last_peered":"2024-09-10T22:57:33.968470+0000","last_clean":"2024-09-10T22:57:33.968470+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:57:33.968470+0000","last_undegraded":"2024-09-10T22:57:33.968470+0000","last_fullsized":"2024-09-10T22:57:33.968470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636127+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:57:33.636127+0000","last_peered":"2024-09-10T22:57:33.636127+0000","last_clean":"2024-09-10T22:57:33.636127+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:57:33.636127+0000","last_undegraded":"2024-09-10T22:57:33.636127+0000","last_fullsized":"2024-09-10T22:57:33.636127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491949+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:57:33.491949+0000","last_peered":"2024-09-10T22:57:33.491949+0000","last_clean":"2024-09-10T22:57:33.491949+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:57:33.491949+0000","last_undegraded":"2024-09-10T22:57:33.491949+0000","last_fullsized":"2024-09-10T22:57:33.491949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:37.483311+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:57:37.483311+0000","last_peered":"2024-09-10T22:57:37.483311+0000","last_clean":"2024-09-10T22:57:37.483311+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:57:37.483311+0000","last_undegraded":"2024-09-10T22:57:37.483311+0000","last_fullsized":"2024-09-10T22:57:37.483311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485425+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:57:33.485425+0000","last_peered":"2024-09-10T22:57:33.485425+0000","last_clean":"2024-09-10T22:57:33.485425+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:57:33.485425+0000","last_undegraded":"2024-09-10T22:57:33.485425+0000","last_fullsized":"2024-09-10T22:57:33.485425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487381+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:57:33.487381+0000","last_peered":"2024-09-10T22:57:33.487381+0000","last_clean":"2024-09-10T22:57:33.487381+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:57:33.487381+0000","last_undegraded":"2024-09-10T22:57:33.487381+0000","last_fullsized":"2024-09-10T22:57:33.487381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484551+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:33.484551+0000","last_peered":"2024-09-10T22:57:33.484551+0000","last_clean":"2024-09-10T22:57:33.484551+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:33.484551+0000","last_undegraded":"2024-09-10T22:57:33.484551+0000","last_fullsized":"2024-09-10T22:57:33.484551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488302+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:33.488302+0000","last_peered":"2024-09-10T22:57:33.488302+0000","last_clean":"2024-09-10T22:57:33.488302+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:33.488302+0000","last_undegraded":"2024-09-10T22:57:33.488302+0000","last_fullsized":"2024-09-10T22:57:33.488302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635806+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:57:33.635806+0000","last_peered":"2024-09-10T22:57:33.635806+0000","last_clean":"2024-09-10T22:57:33.635806+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:57:33.635806+0000","last_undegraded":"2024-09-10T22:57:33.635806+0000","last_fullsized":"2024-09-10T22:57:33.635806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489715+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:33.489715+0000","last_peered":"2024-09-10T22:57:33.489715+0000","last_clean":"2024-09-10T22:57:33.489715+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:33.489715+0000","last_undegraded":"2024-09-10T22:57:33.489715+0000","last_fullsized":"2024-09-10T22:57:33.489715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485286+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:57:33.485286+0000","last_peered":"2024-09-10T22:57:33.485286+0000","last_clean":"2024-09-10T22:57:33.485286+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:57:33.485286+0000","last_undegraded":"2024-09-10T22:57:33.485286+0000","last_fullsized":"2024-09-10T22:57:33.485286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495545+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:57:33.495545+0000","last_peered":"2024-09-10T22:57:33.495545+0000","last_clean":"2024-09-10T22:57:33.495545+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:57:33.495545+0000","last_undegraded":"2024-09-10T22:57:33.495545+0000","last_fullsized":"2024-09-10T22:57:33.495545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:35.495008+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:57:35.495008+0000","last_peered":"2024-09-10T22:57:35.495008+0000","last_clean":"2024-09-10T22:57:35.495008+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:57:35.495008+0000","last_undegraded":"2024-09-10T22:57:35.495008+0000","last_fullsized":"2024-09-10T22:57:35.495008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56669,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":82,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485135+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:57:33.485135+0000","last_peered":"2024-09-10T22:57:33.485135+0000","last_clean":"2024-09-10T22:57:33.485135+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:57:33.485135+0000","last_undegraded":"2024-09-10T22:57:33.485135+0000","last_fullsized":"2024-09-10T22:57:33.485135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484785+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:57:33.484785+0000","last_peered":"2024-09-10T22:57:33.484785+0000","last_clean":"2024-09-10T22:57:33.484785+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:57:33.484785+0000","last_undegraded":"2024-09-10T22:57:33.484785+0000","last_fullsized":"2024-09-10T22:57:33.484785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485645+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:33.485645+0000","last_peered":"2024-09-10T22:57:33.485645+0000","last_clean":"2024-09-10T22:57:33.485645+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:33.485645+0000","last_undegraded":"2024-09-10T22:57:33.485645+0000","last_fullsized":"2024-09-10T22:57:33.485645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933615+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:57:33.933615+0000","last_peered":"2024-09-10T22:57:33.933615+0000","last_clean":"2024-09-10T22:57:33.933615+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:57:33.933615+0000","last_undegraded":"2024-09-10T22:57:33.933615+0000","last_fullsized":"2024-09-10T22:57:33.933615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492566+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:57:33.492566+0000","last_peered":"2024-09-10T22:57:33.492566+0000","last_clean":"2024-09-10T22:57:33.492566+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:57:33.492566+0000","last_undegraded":"2024-09-10T22:57:33.492566+0000","last_fullsized":"2024-09-10T22:57:33.492566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490487+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:33.490487+0000","last_peered":"2024-09-10T22:57:33.490487+0000","last_clean":"2024-09-10T22:57:33.490487+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:33.490487+0000","last_undegraded":"2024-09-10T22:57:33.490487+0000","last_fullsized":"2024-09-10T22:57:33.490487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484599+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:57:33.484599+0000","last_peered":"2024-09-10T22:57:33.484599+0000","last_clean":"2024-09-10T22:57:33.484599+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:57:33.484599+0000","last_undegraded":"2024-09-10T22:57:33.484599+0000","last_fullsized":"2024-09-10T22:57:33.484599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485175+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:57:33.485175+0000","last_peered":"2024-09-10T22:57:33.485175+0000","last_clean":"2024-09-10T22:57:33.485175+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:57:33.485175+0000","last_undegraded":"2024-09-10T22:57:33.485175+0000","last_fullsized":"2024-09-10T22:57:33.485175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:34.495769+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:57:34.495769+0000","last_peered":"2024-09-10T22:57:34.495769+0000","last_clean":"2024-09-10T22:57:34.495769+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:57:34.495769+0000","last_undegraded":"2024-09-10T22:57:34.495769+0000","last_fullsized":"2024-09-10T22:57:34.495769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482848+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:57:33.482848+0000","last_peered":"2024-09-10T22:57:33.482848+0000","last_clean":"2024-09-10T22:57:33.482848+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:57:33.482848+0000","last_undegraded":"2024-09-10T22:57:33.482848+0000","last_fullsized":"2024-09-10T22:57:33.482848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492182+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:57:33.492182+0000","last_peered":"2024-09-10T22:57:33.492182+0000","last_clean":"2024-09-10T22:57:33.492182+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:57:33.492182+0000","last_undegraded":"2024-09-10T22:57:33.492182+0000","last_fullsized":"2024-09-10T22:57:33.492182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484686+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:57:33.484686+0000","last_peered":"2024-09-10T22:57:33.484686+0000","last_clean":"2024-09-10T22:57:33.484686+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:57:33.484686+0000","last_undegraded":"2024-09-10T22:57:33.484686+0000","last_fullsized":"2024-09-10T22:57:33.484686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484688+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:57:33.484688+0000","last_peered":"2024-09-10T22:57:33.484688+0000","last_clean":"2024-09-10T22:57:33.484688+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:57:33.484688+0000","last_undegraded":"2024-09-10T22:57:33.484688+0000","last_fullsized":"2024-09-10T22:57:33.484688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488441+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:33.488441+0000","last_peered":"2024-09-10T22:57:33.488441+0000","last_clean":"2024-09-10T22:57:33.488441+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:33.488441+0000","last_undegraded":"2024-09-10T22:57:33.488441+0000","last_fullsized":"2024-09-10T22:57:33.488441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635694+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:33.635694+0000","last_peered":"2024-09-10T22:57:33.635694+0000","last_clean":"2024-09-10T22:57:33.635694+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:33.635694+0000","last_undegraded":"2024-09-10T22:57:33.635694+0000","last_fullsized":"2024-09-10T22:57:33.635694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489621+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:33.489621+0000","last_peered":"2024-09-10T22:57:33.489621+0000","last_clean":"2024-09-10T22:57:33.489621+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:33.489621+0000","last_undegraded":"2024-09-10T22:57:33.489621+0000","last_fullsized":"2024-09-10T22:57:33.489621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484446+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:57:33.484446+0000","last_peered":"2024-09-10T22:57:33.484446+0000","last_clean":"2024-09-10T22:57:33.484446+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:57:33.484446+0000","last_undegraded":"2024-09-10T22:57:33.484446+0000","last_fullsized":"2024-09-10T22:57:33.484446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489337+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:57:33.489337+0000","last_peered":"2024-09-10T22:57:33.489337+0000","last_clean":"2024-09-10T22:57:33.489337+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:57:33.489337+0000","last_undegraded":"2024-09-10T22:57:33.489337+0000","last_fullsized":"2024-09-10T22:57:33.489337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:37.500568+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:57:37.500568+0000","last_peered":"2024-09-10T22:57:37.500568+0000","last_clean":"2024-09-10T22:57:37.500568+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:57:37.500568+0000","last_undegraded":"2024-09-10T22:57:37.500568+0000","last_fullsized":"2024-09-10T22:57:37.500568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494636+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:57:33.494636+0000","last_peered":"2024-09-10T22:57:33.494636+0000","last_clean":"2024-09-10T22:57:33.494636+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:57:33.494636+0000","last_undegraded":"2024-09-10T22:57:33.494636+0000","last_fullsized":"2024-09-10T22:57:33.494636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490354+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:57:33.490354+0000","last_peered":"2024-09-10T22:57:33.490354+0000","last_clean":"2024-09-10T22:57:33.490354+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:57:33.490354+0000","last_undegraded":"2024-09-10T22:57:33.490354+0000","last_fullsized":"2024-09-10T22:57:33.490354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489485+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:57:33.489485+0000","last_peered":"2024-09-10T22:57:33.489485+0000","last_clean":"2024-09-10T22:57:33.489485+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:57:33.489485+0000","last_undegraded":"2024-09-10T22:57:33.489485+0000","last_fullsized":"2024-09-10T22:57:33.489485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488604+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:57:33.488604+0000","last_peered":"2024-09-10T22:57:33.488604+0000","last_clean":"2024-09-10T22:57:33.488604+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:57:33.488604+0000","last_undegraded":"2024-09-10T22:57:33.488604+0000","last_fullsized":"2024-09-10T22:57:33.488604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488485+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:57:33.488485+0000","last_peered":"2024-09-10T22:57:33.488485+0000","last_clean":"2024-09-10T22:57:33.488485+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:57:33.488485+0000","last_undegraded":"2024-09-10T22:57:33.488485+0000","last_fullsized":"2024-09-10T22:57:33.488485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492236+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:57:33.492236+0000","last_peered":"2024-09-10T22:57:33.492236+0000","last_clean":"2024-09-10T22:57:33.492236+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:57:33.492236+0000","last_undegraded":"2024-09-10T22:57:33.492236+0000","last_fullsized":"2024-09-10T22:57:33.492236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636049+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:57:33.636049+0000","last_peered":"2024-09-10T22:57:33.636049+0000","last_clean":"2024-09-10T22:57:33.636049+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:57:33.636049+0000","last_undegraded":"2024-09-10T22:57:33.636049+0000","last_fullsized":"2024-09-10T22:57:33.636049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488558+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:57:33.488558+0000","last_peered":"2024-09-10T22:57:33.488558+0000","last_clean":"2024-09-10T22:57:33.488558+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:57:33.488558+0000","last_undegraded":"2024-09-10T22:57:33.488558+0000","last_fullsized":"2024-09-10T22:57:33.488558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489253+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:57:33.489253+0000","last_peered":"2024-09-10T22:57:33.489253+0000","last_clean":"2024-09-10T22:57:33.489253+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:57:33.489253+0000","last_undegraded":"2024-09-10T22:57:33.489253+0000","last_fullsized":"2024-09-10T22:57:33.489253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483396+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:57:33.483396+0000","last_peered":"2024-09-10T22:57:33.483396+0000","last_clean":"2024-09-10T22:57:33.483396+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:57:33.483396+0000","last_undegraded":"2024-09-10T22:57:33.483396+0000","last_fullsized":"2024-09-10T22:57:33.483396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484099+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:57:33.484099+0000","last_peered":"2024-09-10T22:57:33.484099+0000","last_clean":"2024-09-10T22:57:33.484099+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:57:33.484099+0000","last_undegraded":"2024-09-10T22:57:33.484099+0000","last_fullsized":"2024-09-10T22:57:33.484099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482935+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:57:33.482935+0000","last_peered":"2024-09-10T22:57:33.482935+0000","last_clean":"2024-09-10T22:57:33.482935+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:57:33.482935+0000","last_undegraded":"2024-09-10T22:57:33.482935+0000","last_fullsized":"2024-09-10T22:57:33.482935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484667+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:57:33.484667+0000","last_peered":"2024-09-10T22:57:33.484667+0000","last_clean":"2024-09-10T22:57:33.484667+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:57:33.484667+0000","last_undegraded":"2024-09-10T22:57:33.484667+0000","last_fullsized":"2024-09-10T22:57:33.484667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635935+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:57:33.635935+0000","last_peered":"2024-09-10T22:57:33.635935+0000","last_clean":"2024-09-10T22:57:33.635935+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:57:33.635935+0000","last_undegraded":"2024-09-10T22:57:33.635935+0000","last_fullsized":"2024-09-10T22:57:33.635935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488423+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:57:33.488423+0000","last_peered":"2024-09-10T22:57:33.488423+0000","last_clean":"2024-09-10T22:57:33.488423+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:57:33.488423+0000","last_undegraded":"2024-09-10T22:57:33.488423+0000","last_fullsized":"2024-09-10T22:57:33.488423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490516+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:57:33.490516+0000","last_peered":"2024-09-10T22:57:33.490516+0000","last_clean":"2024-09-10T22:57:33.490516+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:57:33.490516+0000","last_undegraded":"2024-09-10T22:57:33.490516+0000","last_fullsized":"2024-09-10T22:57:33.490516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485363+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:57:33.485363+0000","last_peered":"2024-09-10T22:57:33.485363+0000","last_clean":"2024-09-10T22:57:33.485363+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:57:33.485363+0000","last_undegraded":"2024-09-10T22:57:33.485363+0000","last_fullsized":"2024-09-10T22:57:33.485363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489898+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:57:33.489898+0000","last_peered":"2024-09-10T22:57:33.489898+0000","last_clean":"2024-09-10T22:57:33.489898+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:57:33.489898+0000","last_undegraded":"2024-09-10T22:57:33.489898+0000","last_fullsized":"2024-09-10T22:57:33.489898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484224+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:57:33.484224+0000","last_peered":"2024-09-10T22:57:33.484224+0000","last_clean":"2024-09-10T22:57:33.484224+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:57:33.484224+0000","last_undegraded":"2024-09-10T22:57:33.484224+0000","last_fullsized":"2024-09-10T22:57:33.484224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491907+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:57:33.491907+0000","last_peered":"2024-09-10T22:57:33.491907+0000","last_clean":"2024-09-10T22:57:33.491907+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:57:33.491907+0000","last_undegraded":"2024-09-10T22:57:33.491907+0000","last_fullsized":"2024-09-10T22:57:33.491907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491892+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:57:33.491892+0000","last_peered":"2024-09-10T22:57:33.491892+0000","last_clean":"2024-09-10T22:57:33.491892+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:57:33.491892+0000","last_undegraded":"2024-09-10T22:57:33.491892+0000","last_fullsized":"2024-09-10T22:57:33.491892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635943+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:57:33.635943+0000","last_peered":"2024-09-10T22:57:33.635943+0000","last_clean":"2024-09-10T22:57:33.635943+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:57:33.635943+0000","last_undegraded":"2024-09-10T22:57:33.635943+0000","last_fullsized":"2024-09-10T22:57:33.635943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934287+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:57:33.934287+0000","last_peered":"2024-09-10T22:57:33.934287+0000","last_clean":"2024-09-10T22:57:33.934287+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:57:33.934287+0000","last_undegraded":"2024-09-10T22:57:33.934287+0000","last_fullsized":"2024-09-10T22:57:33.934287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487645+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:57:33.487645+0000","last_peered":"2024-09-10T22:57:33.487645+0000","last_clean":"2024-09-10T22:57:33.487645+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:57:33.487645+0000","last_undegraded":"2024-09-10T22:57:33.487645+0000","last_fullsized":"2024-09-10T22:57:33.487645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490348+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:57:33.490348+0000","last_peered":"2024-09-10T22:57:33.490348+0000","last_clean":"2024-09-10T22:57:33.490348+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:57:33.490348+0000","last_undegraded":"2024-09-10T22:57:33.490348+0000","last_fullsized":"2024-09-10T22:57:33.490348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485308+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:57:33.485308+0000","last_peered":"2024-09-10T22:57:33.485308+0000","last_clean":"2024-09-10T22:57:33.485308+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:57:33.485308+0000","last_undegraded":"2024-09-10T22:57:33.485308+0000","last_fullsized":"2024-09-10T22:57:33.485308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487369+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:57:33.487369+0000","last_peered":"2024-09-10T22:57:33.487369+0000","last_clean":"2024-09-10T22:57:33.487369+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:57:33.487369+0000","last_undegraded":"2024-09-10T22:57:33.487369+0000","last_fullsized":"2024-09-10T22:57:33.487369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495990+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:57:33.495990+0000","last_peered":"2024-09-10T22:57:33.495990+0000","last_clean":"2024-09-10T22:57:33.495990+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:57:33.495990+0000","last_undegraded":"2024-09-10T22:57:33.495990+0000","last_fullsized":"2024-09-10T22:57:33.495990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494336+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:57:33.494336+0000","last_peered":"2024-09-10T22:57:33.494336+0000","last_clean":"2024-09-10T22:57:33.494336+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:57:33.494336+0000","last_undegraded":"2024-09-10T22:57:33.494336+0000","last_fullsized":"2024-09-10T22:57:33.494336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933856+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:57:33.933856+0000","last_peered":"2024-09-10T22:57:33.933856+0000","last_clean":"2024-09-10T22:57:33.933856+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:57:33.933856+0000","last_undegraded":"2024-09-10T22:57:33.933856+0000","last_fullsized":"2024-09-10T22:57:33.933856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492277+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:57:33.492277+0000","last_peered":"2024-09-10T22:57:33.492277+0000","last_clean":"2024-09-10T22:57:33.492277+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:57:33.492277+0000","last_undegraded":"2024-09-10T22:57:33.492277+0000","last_fullsized":"2024-09-10T22:57:33.492277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492372+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:57:33.492372+0000","last_peered":"2024-09-10T22:57:33.492372+0000","last_clean":"2024-09-10T22:57:33.492372+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:57:33.492372+0000","last_undegraded":"2024-09-10T22:57:33.492372+0000","last_fullsized":"2024-09-10T22:57:33.492372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489777+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:57:33.489777+0000","last_peered":"2024-09-10T22:57:33.489777+0000","last_clean":"2024-09-10T22:57:33.489777+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:57:33.489777+0000","last_undegraded":"2024-09-10T22:57:33.489777+0000","last_fullsized":"2024-09-10T22:57:33.489777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60670,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":174,"num_read_kb":152,"num_write":165,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":192816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612639,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":499051,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":1142876,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972394,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":503271,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":498983,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":1095004,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":494351,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":1090592,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":499113,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":494545,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378710,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":556949,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":557346,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705761,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":495739,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:57:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:42 smithi092 ceph-mon[34223]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:42 smithi067 ceph-mon[30939]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:42 smithi145 ceph-mon[34397]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:43 smithi092 ceph-mon[34223]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 117 B/s, 1 objects/s recovering 2024-09-10T22:57:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:43 smithi067 ceph-mon[30939]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 117 B/s, 1 objects/s recovering 2024-09-10T22:57:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:43 smithi145 ceph-mon[34397]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 117 B/s, 1 objects/s recovering 2024-09-10T22:57:45.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:57:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:45 smithi067 ceph-mon[30939]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 106 B/s, 1 objects/s recovering 2024-09-10T22:57:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:45 smithi145 ceph-mon[34397]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 106 B/s, 1 objects/s recovering 2024-09-10T22:57:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:45 smithi092 ceph-mon[34223]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 106 B/s, 1 objects/s recovering 2024-09-10T22:57:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:46.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 -- 172.21.15.67:0/1316410485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10a800 msgr2=0x7f823c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:46.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 --2- 172.21.15.67:0/1316410485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10a800 0x7f823c10ac40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8224009900 tx=0x7f822402a410 comp rx=0 tx=0).stop 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 -- 172.21.15.67:0/1316410485 shutdown_connections 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 --2- 172.21.15.67:0/1316410485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10b180 0x7f823c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 --2- 172.21.15.67:0/1316410485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10a800 0x7f823c10ac40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 --2- 172.21.15.67:0/1316410485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 0x7f823c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 -- 172.21.15.67:0/1316410485 >> 172.21.15.67:0/1316410485 conn(0x7f823c100840 msgr2=0x7f823c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 -- 172.21.15.67:0/1316410485 shutdown_connections 2024-09-10T22:57:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.706+0000 7f8241600640 1 -- 172.21.15.67:0/1316410485 wait complete. 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.707+0000 7f8241600640 1 Processor -- start 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.707+0000 7f8241600640 1 -- start start 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.707+0000 7f8241600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 0x7f823c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.707+0000 7f8241600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10a800 0x7f823c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8241600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8241600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f823c114cf0 con 0x7f823c10b180 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8241600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f823c114b70 con 0x7f823c10a800 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8241600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f823c114e70 con 0x7f823c104df0 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36026/0 (socket says 172.21.15.67:36026) 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 -- 172.21.15.67:0/318116019 learned_addr learned my addr 172.21.15.67:0/318116019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f823be00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 0x7f823c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8233400640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10a800 0x7f823c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 -- 172.21.15.67:0/318116019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 msgr2=0x7f823c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 0x7f823c19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 -- 172.21.15.67:0/318116019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10a800 msgr2=0x7f823c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10a800 0x7f823c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.708+0000 7f8240c00640 1 -- 172.21.15.67:0/318116019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1a1250 con 0x7f823c10b180 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8240c00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f823400e970 tx=0x7f823400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400ccc0 con 0x7f823c10b180 2024-09-10T22:57:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823400ce60 con 0x7f823c10b180 2024-09-10T22:57:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234010430 con 0x7f823c10b180 2024-09-10T22:57:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1a1540 con 0x7f823c10b180 2024-09-10T22:57:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.709+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a7c00 con 0x7f823c10b180 2024-09-10T22:57:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.710+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c1051d0 con 0x7f823c10b180 2024-09-10T22:57:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.712+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f8234004f10 con 0x7f823c10b180 2024-09-10T22:57:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.714+0000 7f8239600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 0x7f82200b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.714+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f82340e6620 con 0x7f823c10b180 2024-09-10T22:57:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.714+0000 7f823be00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 0x7f82200b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.715+0000 7f823be00640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 0x7f82200b79a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f822c005fd0 tx=0x7f822c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.718+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f82340acf00 con 0x7f823c10b180 2024-09-10T22:57:46.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.910+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f823c063590 con 0x7f82200b54e0 2024-09-10T22:57:46.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.916+0000 7f8239600640 1 -- 172.21.15.67:0/318116019 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340627 (secure 0 0 0) 0x7f823c063590 con 0x7f82200b54e0 2024-09-10T22:57:46.921 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:46.923 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.920+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 msgr2=0x7f82200b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.920+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 0x7f82200b79a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f822c005fd0 tx=0x7f822c009500 comp rx=0 tx=0).stop 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.920+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 msgr2=0x7f823c1a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.920+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f823400e970 tx=0x7f823400ed70 comp rx=0 tx=0).stop 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 shutdown_connections 2024-09-10T22:57:46.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f82200b54e0 0x7f82200b79a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f823c10b180 0x7f823c1a0a80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c10a800 0x7f823c19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 --2- 172.21.15.67:0/318116019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c104df0 0x7f823c19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 >> 172.21.15.67:0/318116019 conn(0x7f823c100840 msgr2=0x7f823c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 shutdown_connections 2024-09-10T22:57:46.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:46.921+0000 7f8241600640 1 -- 172.21.15.67:0/318116019 wait complete. 2024-09-10T22:57:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:47 smithi067 ceph-mon[30939]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:47 smithi067 ceph-mon[30939]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:47.859 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T22:57:46.079143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656675,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":301,"num_read_kb":252,"num_write":293,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356816,"kb_used_data":11672,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585008,"statfs":{"total":1151940427776,"available":1151575048192,"internally_reserved":0,"allocated":11952128,"data_stored":7933223,"data_compressed":391371,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":55,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003785"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485580+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:33.485580+0000","last_peered":"2024-09-10T22:57:33.485580+0000","last_clean":"2024-09-10T22:57:33.485580+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:33.485580+0000","last_undegraded":"2024-09-10T22:57:33.485580+0000","last_fullsized":"2024-09-10T22:57:33.485580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483317+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:57:33.483317+0000","last_peered":"2024-09-10T22:57:33.483317+0000","last_clean":"2024-09-10T22:57:33.483317+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:57:33.483317+0000","last_undegraded":"2024-09-10T22:57:33.483317+0000","last_fullsized":"2024-09-10T22:57:33.483317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488147+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:57:33.488147+0000","last_peered":"2024-09-10T22:57:33.488147+0000","last_clean":"2024-09-10T22:57:33.488147+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:57:33.488147+0000","last_undegraded":"2024-09-10T22:57:33.488147+0000","last_fullsized":"2024-09-10T22:57:33.488147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483508+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:57:33.483508+0000","last_peered":"2024-09-10T22:57:33.483508+0000","last_clean":"2024-09-10T22:57:33.483508+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:57:33.483508+0000","last_undegraded":"2024-09-10T22:57:33.483508+0000","last_fullsized":"2024-09-10T22:57:33.483508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492567+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:57:33.492567+0000","last_peered":"2024-09-10T22:57:33.492567+0000","last_clean":"2024-09-10T22:57:33.492567+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:57:33.492567+0000","last_undegraded":"2024-09-10T22:57:33.492567+0000","last_fullsized":"2024-09-10T22:57:33.492567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933763+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:57:33.933763+0000","last_peered":"2024-09-10T22:57:33.933763+0000","last_clean":"2024-09-10T22:57:33.933763+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:57:33.933763+0000","last_undegraded":"2024-09-10T22:57:33.933763+0000","last_fullsized":"2024-09-10T22:57:33.933763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495695+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:33.495695+0000","last_peered":"2024-09-10T22:57:33.495695+0000","last_clean":"2024-09-10T22:57:33.495695+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:33.495695+0000","last_undegraded":"2024-09-10T22:57:33.495695+0000","last_fullsized":"2024-09-10T22:57:33.495695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484689+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:57:33.484689+0000","last_peered":"2024-09-10T22:57:33.484689+0000","last_clean":"2024-09-10T22:57:33.484689+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:57:33.484689+0000","last_undegraded":"2024-09-10T22:57:33.484689+0000","last_fullsized":"2024-09-10T22:57:33.484689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933235+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:57:33.933235+0000","last_peered":"2024-09-10T22:57:33.933235+0000","last_clean":"2024-09-10T22:57:33.933235+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:57:33.933235+0000","last_undegraded":"2024-09-10T22:57:33.933235+0000","last_fullsized":"2024-09-10T22:57:33.933235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.499755+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:57:33.499755+0000","last_peered":"2024-09-10T22:57:33.499755+0000","last_clean":"2024-09-10T22:57:33.499755+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:57:33.499755+0000","last_undegraded":"2024-09-10T22:57:33.499755+0000","last_fullsized":"2024-09-10T22:57:33.499755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.637598+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:57:33.637598+0000","last_peered":"2024-09-10T22:57:33.637598+0000","last_clean":"2024-09-10T22:57:33.637598+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:57:33.637598+0000","last_undegraded":"2024-09-10T22:57:33.637598+0000","last_fullsized":"2024-09-10T22:57:33.637598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933169+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:57:33.933169+0000","last_peered":"2024-09-10T22:57:33.933169+0000","last_clean":"2024-09-10T22:57:33.933169+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:57:33.933169+0000","last_undegraded":"2024-09-10T22:57:33.933169+0000","last_fullsized":"2024-09-10T22:57:33.933169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489400+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:57:33.489400+0000","last_peered":"2024-09-10T22:57:33.489400+0000","last_clean":"2024-09-10T22:57:33.489400+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:57:33.489400+0000","last_undegraded":"2024-09-10T22:57:33.489400+0000","last_fullsized":"2024-09-10T22:57:33.489400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494981+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:33.494981+0000","last_peered":"2024-09-10T22:57:33.494981+0000","last_clean":"2024-09-10T22:57:33.494981+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:33.494981+0000","last_undegraded":"2024-09-10T22:57:33.494981+0000","last_fullsized":"2024-09-10T22:57:33.494981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933680+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:57:33.933680+0000","last_peered":"2024-09-10T22:57:33.933680+0000","last_clean":"2024-09-10T22:57:33.933680+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:57:33.933680+0000","last_undegraded":"2024-09-10T22:57:33.933680+0000","last_fullsized":"2024-09-10T22:57:33.933680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483072+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:57:33.483072+0000","last_peered":"2024-09-10T22:57:33.483072+0000","last_clean":"2024-09-10T22:57:33.483072+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:57:33.483072+0000","last_undegraded":"2024-09-10T22:57:33.483072+0000","last_fullsized":"2024-09-10T22:57:33.483072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488646+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:57:33.488646+0000","last_peered":"2024-09-10T22:57:33.488646+0000","last_clean":"2024-09-10T22:57:33.488646+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:57:33.488646+0000","last_undegraded":"2024-09-10T22:57:33.488646+0000","last_fullsized":"2024-09-10T22:57:33.488646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933688+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:57:33.933688+0000","last_peered":"2024-09-10T22:57:33.933688+0000","last_clean":"2024-09-10T22:57:33.933688+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:57:33.933688+0000","last_undegraded":"2024-09-10T22:57:33.933688+0000","last_fullsized":"2024-09-10T22:57:33.933688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489595+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:57:33.489595+0000","last_peered":"2024-09-10T22:57:33.489595+0000","last_clean":"2024-09-10T22:57:33.489595+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:57:33.489595+0000","last_undegraded":"2024-09-10T22:57:33.489595+0000","last_fullsized":"2024-09-10T22:57:33.489595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482999+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:33.482999+0000","last_peered":"2024-09-10T22:57:33.482999+0000","last_clean":"2024-09-10T22:57:33.482999+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:33.482999+0000","last_undegraded":"2024-09-10T22:57:33.482999+0000","last_fullsized":"2024-09-10T22:57:33.482999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485365+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:57:33.485365+0000","last_peered":"2024-09-10T22:57:33.485365+0000","last_clean":"2024-09-10T22:57:33.485365+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:57:33.485365+0000","last_undegraded":"2024-09-10T22:57:33.485365+0000","last_fullsized":"2024-09-10T22:57:33.485365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493982+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:57:33.493982+0000","last_peered":"2024-09-10T22:57:33.493982+0000","last_clean":"2024-09-10T22:57:33.493982+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:57:33.493982+0000","last_undegraded":"2024-09-10T22:57:33.493982+0000","last_fullsized":"2024-09-10T22:57:33.493982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484357+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:33.484357+0000","last_peered":"2024-09-10T22:57:33.484357+0000","last_clean":"2024-09-10T22:57:33.484357+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:33.484357+0000","last_undegraded":"2024-09-10T22:57:33.484357+0000","last_fullsized":"2024-09-10T22:57:33.484357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933699+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:57:33.933699+0000","last_peered":"2024-09-10T22:57:33.933699+0000","last_clean":"2024-09-10T22:57:33.933699+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:57:33.933699+0000","last_undegraded":"2024-09-10T22:57:33.933699+0000","last_fullsized":"2024-09-10T22:57:33.933699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492434+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:57:33.492434+0000","last_peered":"2024-09-10T22:57:33.492434+0000","last_clean":"2024-09-10T22:57:33.492434+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:57:33.492434+0000","last_undegraded":"2024-09-10T22:57:33.492434+0000","last_fullsized":"2024-09-10T22:57:33.492434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934281+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:33.934281+0000","last_peered":"2024-09-10T22:57:33.934281+0000","last_clean":"2024-09-10T22:57:33.934281+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:33.934281+0000","last_undegraded":"2024-09-10T22:57:33.934281+0000","last_fullsized":"2024-09-10T22:57:33.934281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488401+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:57:33.488401+0000","last_peered":"2024-09-10T22:57:33.488401+0000","last_clean":"2024-09-10T22:57:33.488401+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:57:33.488401+0000","last_undegraded":"2024-09-10T22:57:33.488401+0000","last_fullsized":"2024-09-10T22:57:33.488401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483447+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:57:33.483447+0000","last_peered":"2024-09-10T22:57:33.483447+0000","last_clean":"2024-09-10T22:57:33.483447+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:57:33.483447+0000","last_undegraded":"2024-09-10T22:57:33.483447+0000","last_fullsized":"2024-09-10T22:57:33.483447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485392+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:33.485392+0000","last_peered":"2024-09-10T22:57:33.485392+0000","last_clean":"2024-09-10T22:57:33.485392+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:33.485392+0000","last_undegraded":"2024-09-10T22:57:33.485392+0000","last_fullsized":"2024-09-10T22:57:33.485392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482709+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:57:33.482709+0000","last_peered":"2024-09-10T22:57:33.482709+0000","last_clean":"2024-09-10T22:57:33.482709+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:57:33.482709+0000","last_undegraded":"2024-09-10T22:57:33.482709+0000","last_fullsized":"2024-09-10T22:57:33.482709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636052+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:57:33.636052+0000","last_peered":"2024-09-10T22:57:33.636052+0000","last_clean":"2024-09-10T22:57:33.636052+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:57:33.636052+0000","last_undegraded":"2024-09-10T22:57:33.636052+0000","last_fullsized":"2024-09-10T22:57:33.636052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495704+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:33.495704+0000","last_peered":"2024-09-10T22:57:33.495704+0000","last_clean":"2024-09-10T22:57:33.495704+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:33.495704+0000","last_undegraded":"2024-09-10T22:57:33.495704+0000","last_fullsized":"2024-09-10T22:57:33.495704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487731+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:33.487731+0000","last_peered":"2024-09-10T22:57:33.487731+0000","last_clean":"2024-09-10T22:57:33.487731+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:33.487731+0000","last_undegraded":"2024-09-10T22:57:33.487731+0000","last_fullsized":"2024-09-10T22:57:33.487731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489113+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:57:33.489113+0000","last_peered":"2024-09-10T22:57:33.489113+0000","last_clean":"2024-09-10T22:57:33.489113+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:57:33.489113+0000","last_undegraded":"2024-09-10T22:57:33.489113+0000","last_fullsized":"2024-09-10T22:57:33.489113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485163+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:57:33.485163+0000","last_peered":"2024-09-10T22:57:33.485163+0000","last_clean":"2024-09-10T22:57:33.485163+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:57:33.485163+0000","last_undegraded":"2024-09-10T22:57:33.485163+0000","last_fullsized":"2024-09-10T22:57:33.485163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493448+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:57:33.493448+0000","last_peered":"2024-09-10T22:57:33.493448+0000","last_clean":"2024-09-10T22:57:33.493448+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:57:33.493448+0000","last_undegraded":"2024-09-10T22:57:33.493448+0000","last_fullsized":"2024-09-10T22:57:33.493448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490092+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:57:33.490092+0000","last_peered":"2024-09-10T22:57:33.490092+0000","last_clean":"2024-09-10T22:57:33.490092+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:57:33.490092+0000","last_undegraded":"2024-09-10T22:57:33.490092+0000","last_fullsized":"2024-09-10T22:57:33.490092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489599+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:57:33.489599+0000","last_peered":"2024-09-10T22:57:33.489599+0000","last_clean":"2024-09-10T22:57:33.489599+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:57:33.489599+0000","last_undegraded":"2024-09-10T22:57:33.489599+0000","last_fullsized":"2024-09-10T22:57:33.489599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635906+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:57:33.635906+0000","last_peered":"2024-09-10T22:57:33.635906+0000","last_clean":"2024-09-10T22:57:33.635906+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:57:33.635906+0000","last_undegraded":"2024-09-10T22:57:33.635906+0000","last_fullsized":"2024-09-10T22:57:33.635906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484247+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:57:33.484247+0000","last_peered":"2024-09-10T22:57:33.484247+0000","last_clean":"2024-09-10T22:57:33.484247+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:57:33.484247+0000","last_undegraded":"2024-09-10T22:57:33.484247+0000","last_fullsized":"2024-09-10T22:57:33.484247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:43.494592+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:57:43.494592+0000","last_peered":"2024-09-10T22:57:43.494592+0000","last_clean":"2024-09-10T22:57:43.494592+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:57:43.494592+0000","last_undegraded":"2024-09-10T22:57:43.494592+0000","last_fullsized":"2024-09-10T22:57:43.494592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636145+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:57:33.636145+0000","last_peered":"2024-09-10T22:57:33.636145+0000","last_clean":"2024-09-10T22:57:33.636145+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:57:33.636145+0000","last_undegraded":"2024-09-10T22:57:33.636145+0000","last_fullsized":"2024-09-10T22:57:33.636145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635652+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:57:33.635652+0000","last_peered":"2024-09-10T22:57:33.635652+0000","last_clean":"2024-09-10T22:57:33.635652+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:57:33.635652+0000","last_undegraded":"2024-09-10T22:57:33.635652+0000","last_fullsized":"2024-09-10T22:57:33.635652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487480+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:33.487480+0000","last_peered":"2024-09-10T22:57:33.487480+0000","last_clean":"2024-09-10T22:57:33.487480+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:33.487480+0000","last_undegraded":"2024-09-10T22:57:33.487480+0000","last_fullsized":"2024-09-10T22:57:33.487480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484718+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:57:33.484718+0000","last_peered":"2024-09-10T22:57:33.484718+0000","last_clean":"2024-09-10T22:57:33.484718+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:57:33.484718+0000","last_undegraded":"2024-09-10T22:57:33.484718+0000","last_fullsized":"2024-09-10T22:57:33.484718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489346+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:57:33.489346+0000","last_peered":"2024-09-10T22:57:33.489346+0000","last_clean":"2024-09-10T22:57:33.489346+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:57:33.489346+0000","last_undegraded":"2024-09-10T22:57:33.489346+0000","last_fullsized":"2024-09-10T22:57:33.489346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934139+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:33.934139+0000","last_peered":"2024-09-10T22:57:33.934139+0000","last_clean":"2024-09-10T22:57:33.934139+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:33.934139+0000","last_undegraded":"2024-09-10T22:57:33.934139+0000","last_fullsized":"2024-09-10T22:57:33.934139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487351+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:57:33.487351+0000","last_peered":"2024-09-10T22:57:33.487351+0000","last_clean":"2024-09-10T22:57:33.487351+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:57:33.487351+0000","last_undegraded":"2024-09-10T22:57:33.487351+0000","last_fullsized":"2024-09-10T22:57:33.487351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487434+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:57:33.487434+0000","last_peered":"2024-09-10T22:57:33.487434+0000","last_clean":"2024-09-10T22:57:33.487434+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:57:33.487434+0000","last_undegraded":"2024-09-10T22:57:33.487434+0000","last_fullsized":"2024-09-10T22:57:33.487434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487965+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:57:33.487965+0000","last_peered":"2024-09-10T22:57:33.487965+0000","last_clean":"2024-09-10T22:57:33.487965+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:57:33.487965+0000","last_undegraded":"2024-09-10T22:57:33.487965+0000","last_fullsized":"2024-09-10T22:57:33.487965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489704+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:33.489704+0000","last_peered":"2024-09-10T22:57:33.489704+0000","last_clean":"2024-09-10T22:57:33.489704+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:33.489704+0000","last_undegraded":"2024-09-10T22:57:33.489704+0000","last_fullsized":"2024-09-10T22:57:33.489704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484515+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:57:33.484515+0000","last_peered":"2024-09-10T22:57:33.484515+0000","last_clean":"2024-09-10T22:57:33.484515+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:57:33.484515+0000","last_undegraded":"2024-09-10T22:57:33.484515+0000","last_fullsized":"2024-09-10T22:57:33.484515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495905+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:57:33.495905+0000","last_peered":"2024-09-10T22:57:33.495905+0000","last_clean":"2024-09-10T22:57:33.495905+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:57:33.495905+0000","last_undegraded":"2024-09-10T22:57:33.495905+0000","last_fullsized":"2024-09-10T22:57:33.495905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485826+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:57:33.485826+0000","last_peered":"2024-09-10T22:57:33.485826+0000","last_clean":"2024-09-10T22:57:33.485826+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:57:33.485826+0000","last_undegraded":"2024-09-10T22:57:33.485826+0000","last_fullsized":"2024-09-10T22:57:33.485826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492168+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:57:33.492168+0000","last_peered":"2024-09-10T22:57:33.492168+0000","last_clean":"2024-09-10T22:57:33.492168+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:57:33.492168+0000","last_undegraded":"2024-09-10T22:57:33.492168+0000","last_fullsized":"2024-09-10T22:57:33.492168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487627+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:57:33.487627+0000","last_peered":"2024-09-10T22:57:33.487627+0000","last_clean":"2024-09-10T22:57:33.487627+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:57:33.487627+0000","last_undegraded":"2024-09-10T22:57:33.487627+0000","last_fullsized":"2024-09-10T22:57:33.487627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636233+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:57:33.636233+0000","last_peered":"2024-09-10T22:57:33.636233+0000","last_clean":"2024-09-10T22:57:33.636233+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:57:33.636233+0000","last_undegraded":"2024-09-10T22:57:33.636233+0000","last_fullsized":"2024-09-10T22:57:33.636233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483887+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:57:33.483887+0000","last_peered":"2024-09-10T22:57:33.483887+0000","last_clean":"2024-09-10T22:57:33.483887+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:57:33.483887+0000","last_undegraded":"2024-09-10T22:57:33.483887+0000","last_fullsized":"2024-09-10T22:57:33.483887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489318+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:57:33.489318+0000","last_peered":"2024-09-10T22:57:33.489318+0000","last_clean":"2024-09-10T22:57:33.489318+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:57:33.489318+0000","last_undegraded":"2024-09-10T22:57:33.489318+0000","last_fullsized":"2024-09-10T22:57:33.489318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495716+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:33.495716+0000","last_peered":"2024-09-10T22:57:33.495716+0000","last_clean":"2024-09-10T22:57:33.495716+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:33.495716+0000","last_undegraded":"2024-09-10T22:57:33.495716+0000","last_fullsized":"2024-09-10T22:57:33.495716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484068+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:57:33.484068+0000","last_peered":"2024-09-10T22:57:33.484068+0000","last_clean":"2024-09-10T22:57:33.484068+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:57:33.484068+0000","last_undegraded":"2024-09-10T22:57:33.484068+0000","last_fullsized":"2024-09-10T22:57:33.484068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490385+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:57:33.490385+0000","last_peered":"2024-09-10T22:57:33.490385+0000","last_clean":"2024-09-10T22:57:33.490385+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:57:33.490385+0000","last_undegraded":"2024-09-10T22:57:33.490385+0000","last_fullsized":"2024-09-10T22:57:33.490385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.968470+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:57:33.968470+0000","last_peered":"2024-09-10T22:57:33.968470+0000","last_clean":"2024-09-10T22:57:33.968470+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:57:33.968470+0000","last_undegraded":"2024-09-10T22:57:33.968470+0000","last_fullsized":"2024-09-10T22:57:33.968470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636127+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:57:33.636127+0000","last_peered":"2024-09-10T22:57:33.636127+0000","last_clean":"2024-09-10T22:57:33.636127+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:57:33.636127+0000","last_undegraded":"2024-09-10T22:57:33.636127+0000","last_fullsized":"2024-09-10T22:57:33.636127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491949+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:57:33.491949+0000","last_peered":"2024-09-10T22:57:33.491949+0000","last_clean":"2024-09-10T22:57:33.491949+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:57:33.491949+0000","last_undegraded":"2024-09-10T22:57:33.491949+0000","last_fullsized":"2024-09-10T22:57:33.491949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:42.493392+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:57:42.493392+0000","last_peered":"2024-09-10T22:57:42.493392+0000","last_clean":"2024-09-10T22:57:42.493392+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:57:42.493392+0000","last_undegraded":"2024-09-10T22:57:42.493392+0000","last_fullsized":"2024-09-10T22:57:42.493392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485425+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:57:33.485425+0000","last_peered":"2024-09-10T22:57:33.485425+0000","last_clean":"2024-09-10T22:57:33.485425+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:57:33.485425+0000","last_undegraded":"2024-09-10T22:57:33.485425+0000","last_fullsized":"2024-09-10T22:57:33.485425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487381+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:57:33.487381+0000","last_peered":"2024-09-10T22:57:33.487381+0000","last_clean":"2024-09-10T22:57:33.487381+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:57:33.487381+0000","last_undegraded":"2024-09-10T22:57:33.487381+0000","last_fullsized":"2024-09-10T22:57:33.487381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484551+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:33.484551+0000","last_peered":"2024-09-10T22:57:33.484551+0000","last_clean":"2024-09-10T22:57:33.484551+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:33.484551+0000","last_undegraded":"2024-09-10T22:57:33.484551+0000","last_fullsized":"2024-09-10T22:57:33.484551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488302+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:33.488302+0000","last_peered":"2024-09-10T22:57:33.488302+0000","last_clean":"2024-09-10T22:57:33.488302+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:33.488302+0000","last_undegraded":"2024-09-10T22:57:33.488302+0000","last_fullsized":"2024-09-10T22:57:33.488302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635806+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:57:33.635806+0000","last_peered":"2024-09-10T22:57:33.635806+0000","last_clean":"2024-09-10T22:57:33.635806+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:57:33.635806+0000","last_undegraded":"2024-09-10T22:57:33.635806+0000","last_fullsized":"2024-09-10T22:57:33.635806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489715+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:33.489715+0000","last_peered":"2024-09-10T22:57:33.489715+0000","last_clean":"2024-09-10T22:57:33.489715+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:33.489715+0000","last_undegraded":"2024-09-10T22:57:33.489715+0000","last_fullsized":"2024-09-10T22:57:33.489715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485286+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:57:33.485286+0000","last_peered":"2024-09-10T22:57:33.485286+0000","last_clean":"2024-09-10T22:57:33.485286+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:57:33.485286+0000","last_undegraded":"2024-09-10T22:57:33.485286+0000","last_fullsized":"2024-09-10T22:57:33.485286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495545+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:57:33.495545+0000","last_peered":"2024-09-10T22:57:33.495545+0000","last_clean":"2024-09-10T22:57:33.495545+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:57:33.495545+0000","last_undegraded":"2024-09-10T22:57:33.495545+0000","last_fullsized":"2024-09-10T22:57:33.495545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:45.512079+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:57:45.512079+0000","last_peered":"2024-09-10T22:57:45.512079+0000","last_clean":"2024-09-10T22:57:45.512079+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:57:45.512079+0000","last_undegraded":"2024-09-10T22:57:45.512079+0000","last_fullsized":"2024-09-10T22:57:45.512079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":98,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485135+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:57:33.485135+0000","last_peered":"2024-09-10T22:57:33.485135+0000","last_clean":"2024-09-10T22:57:33.485135+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:57:33.485135+0000","last_undegraded":"2024-09-10T22:57:33.485135+0000","last_fullsized":"2024-09-10T22:57:33.485135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484785+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:57:33.484785+0000","last_peered":"2024-09-10T22:57:33.484785+0000","last_clean":"2024-09-10T22:57:33.484785+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:57:33.484785+0000","last_undegraded":"2024-09-10T22:57:33.484785+0000","last_fullsized":"2024-09-10T22:57:33.484785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485645+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:33.485645+0000","last_peered":"2024-09-10T22:57:33.485645+0000","last_clean":"2024-09-10T22:57:33.485645+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:33.485645+0000","last_undegraded":"2024-09-10T22:57:33.485645+0000","last_fullsized":"2024-09-10T22:57:33.485645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933615+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:57:33.933615+0000","last_peered":"2024-09-10T22:57:33.933615+0000","last_clean":"2024-09-10T22:57:33.933615+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:57:33.933615+0000","last_undegraded":"2024-09-10T22:57:33.933615+0000","last_fullsized":"2024-09-10T22:57:33.933615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492566+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:57:33.492566+0000","last_peered":"2024-09-10T22:57:33.492566+0000","last_clean":"2024-09-10T22:57:33.492566+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:57:33.492566+0000","last_undegraded":"2024-09-10T22:57:33.492566+0000","last_fullsized":"2024-09-10T22:57:33.492566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490487+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:33.490487+0000","last_peered":"2024-09-10T22:57:33.490487+0000","last_clean":"2024-09-10T22:57:33.490487+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:33.490487+0000","last_undegraded":"2024-09-10T22:57:33.490487+0000","last_fullsized":"2024-09-10T22:57:33.490487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484599+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:57:33.484599+0000","last_peered":"2024-09-10T22:57:33.484599+0000","last_clean":"2024-09-10T22:57:33.484599+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:57:33.484599+0000","last_undegraded":"2024-09-10T22:57:33.484599+0000","last_fullsized":"2024-09-10T22:57:33.484599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485175+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:57:33.485175+0000","last_peered":"2024-09-10T22:57:33.485175+0000","last_clean":"2024-09-10T22:57:33.485175+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:57:33.485175+0000","last_undegraded":"2024-09-10T22:57:33.485175+0000","last_fullsized":"2024-09-10T22:57:33.485175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:44.510403+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:57:44.510403+0000","last_peered":"2024-09-10T22:57:44.510403+0000","last_clean":"2024-09-10T22:57:44.510403+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:57:44.510403+0000","last_undegraded":"2024-09-10T22:57:44.510403+0000","last_fullsized":"2024-09-10T22:57:44.510403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482848+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:57:33.482848+0000","last_peered":"2024-09-10T22:57:33.482848+0000","last_clean":"2024-09-10T22:57:33.482848+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:57:33.482848+0000","last_undegraded":"2024-09-10T22:57:33.482848+0000","last_fullsized":"2024-09-10T22:57:33.482848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492182+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:57:33.492182+0000","last_peered":"2024-09-10T22:57:33.492182+0000","last_clean":"2024-09-10T22:57:33.492182+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:57:33.492182+0000","last_undegraded":"2024-09-10T22:57:33.492182+0000","last_fullsized":"2024-09-10T22:57:33.492182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484686+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:57:33.484686+0000","last_peered":"2024-09-10T22:57:33.484686+0000","last_clean":"2024-09-10T22:57:33.484686+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:57:33.484686+0000","last_undegraded":"2024-09-10T22:57:33.484686+0000","last_fullsized":"2024-09-10T22:57:33.484686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484688+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:57:33.484688+0000","last_peered":"2024-09-10T22:57:33.484688+0000","last_clean":"2024-09-10T22:57:33.484688+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:57:33.484688+0000","last_undegraded":"2024-09-10T22:57:33.484688+0000","last_fullsized":"2024-09-10T22:57:33.484688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488441+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:33.488441+0000","last_peered":"2024-09-10T22:57:33.488441+0000","last_clean":"2024-09-10T22:57:33.488441+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:33.488441+0000","last_undegraded":"2024-09-10T22:57:33.488441+0000","last_fullsized":"2024-09-10T22:57:33.488441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635694+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:33.635694+0000","last_peered":"2024-09-10T22:57:33.635694+0000","last_clean":"2024-09-10T22:57:33.635694+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:33.635694+0000","last_undegraded":"2024-09-10T22:57:33.635694+0000","last_fullsized":"2024-09-10T22:57:33.635694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489621+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:33.489621+0000","last_peered":"2024-09-10T22:57:33.489621+0000","last_clean":"2024-09-10T22:57:33.489621+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:33.489621+0000","last_undegraded":"2024-09-10T22:57:33.489621+0000","last_fullsized":"2024-09-10T22:57:33.489621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484446+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:57:33.484446+0000","last_peered":"2024-09-10T22:57:33.484446+0000","last_clean":"2024-09-10T22:57:33.484446+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:57:33.484446+0000","last_undegraded":"2024-09-10T22:57:33.484446+0000","last_fullsized":"2024-09-10T22:57:33.484446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489337+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:57:33.489337+0000","last_peered":"2024-09-10T22:57:33.489337+0000","last_clean":"2024-09-10T22:57:33.489337+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:57:33.489337+0000","last_undegraded":"2024-09-10T22:57:33.489337+0000","last_fullsized":"2024-09-10T22:57:33.489337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:42.507926+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:57:42.507926+0000","last_peered":"2024-09-10T22:57:42.507926+0000","last_clean":"2024-09-10T22:57:42.507926+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:57:42.507926+0000","last_undegraded":"2024-09-10T22:57:42.507926+0000","last_fullsized":"2024-09-10T22:57:42.507926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494636+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:57:33.494636+0000","last_peered":"2024-09-10T22:57:33.494636+0000","last_clean":"2024-09-10T22:57:33.494636+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:57:33.494636+0000","last_undegraded":"2024-09-10T22:57:33.494636+0000","last_fullsized":"2024-09-10T22:57:33.494636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490354+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:57:33.490354+0000","last_peered":"2024-09-10T22:57:33.490354+0000","last_clean":"2024-09-10T22:57:33.490354+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:57:33.490354+0000","last_undegraded":"2024-09-10T22:57:33.490354+0000","last_fullsized":"2024-09-10T22:57:33.490354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489485+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:57:33.489485+0000","last_peered":"2024-09-10T22:57:33.489485+0000","last_clean":"2024-09-10T22:57:33.489485+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:57:33.489485+0000","last_undegraded":"2024-09-10T22:57:33.489485+0000","last_fullsized":"2024-09-10T22:57:33.489485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488604+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:57:33.488604+0000","last_peered":"2024-09-10T22:57:33.488604+0000","last_clean":"2024-09-10T22:57:33.488604+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:57:33.488604+0000","last_undegraded":"2024-09-10T22:57:33.488604+0000","last_fullsized":"2024-09-10T22:57:33.488604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488485+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:57:33.488485+0000","last_peered":"2024-09-10T22:57:33.488485+0000","last_clean":"2024-09-10T22:57:33.488485+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:57:33.488485+0000","last_undegraded":"2024-09-10T22:57:33.488485+0000","last_fullsized":"2024-09-10T22:57:33.488485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492236+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:57:33.492236+0000","last_peered":"2024-09-10T22:57:33.492236+0000","last_clean":"2024-09-10T22:57:33.492236+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:57:33.492236+0000","last_undegraded":"2024-09-10T22:57:33.492236+0000","last_fullsized":"2024-09-10T22:57:33.492236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636049+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:57:33.636049+0000","last_peered":"2024-09-10T22:57:33.636049+0000","last_clean":"2024-09-10T22:57:33.636049+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:57:33.636049+0000","last_undegraded":"2024-09-10T22:57:33.636049+0000","last_fullsized":"2024-09-10T22:57:33.636049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488558+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:57:33.488558+0000","last_peered":"2024-09-10T22:57:33.488558+0000","last_clean":"2024-09-10T22:57:33.488558+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:57:33.488558+0000","last_undegraded":"2024-09-10T22:57:33.488558+0000","last_fullsized":"2024-09-10T22:57:33.488558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489253+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:57:33.489253+0000","last_peered":"2024-09-10T22:57:33.489253+0000","last_clean":"2024-09-10T22:57:33.489253+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:57:33.489253+0000","last_undegraded":"2024-09-10T22:57:33.489253+0000","last_fullsized":"2024-09-10T22:57:33.489253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483396+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:57:33.483396+0000","last_peered":"2024-09-10T22:57:33.483396+0000","last_clean":"2024-09-10T22:57:33.483396+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:57:33.483396+0000","last_undegraded":"2024-09-10T22:57:33.483396+0000","last_fullsized":"2024-09-10T22:57:33.483396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484099+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:57:33.484099+0000","last_peered":"2024-09-10T22:57:33.484099+0000","last_clean":"2024-09-10T22:57:33.484099+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:57:33.484099+0000","last_undegraded":"2024-09-10T22:57:33.484099+0000","last_fullsized":"2024-09-10T22:57:33.484099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482935+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:57:33.482935+0000","last_peered":"2024-09-10T22:57:33.482935+0000","last_clean":"2024-09-10T22:57:33.482935+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:57:33.482935+0000","last_undegraded":"2024-09-10T22:57:33.482935+0000","last_fullsized":"2024-09-10T22:57:33.482935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484667+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:57:33.484667+0000","last_peered":"2024-09-10T22:57:33.484667+0000","last_clean":"2024-09-10T22:57:33.484667+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:57:33.484667+0000","last_undegraded":"2024-09-10T22:57:33.484667+0000","last_fullsized":"2024-09-10T22:57:33.484667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635935+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:57:33.635935+0000","last_peered":"2024-09-10T22:57:33.635935+0000","last_clean":"2024-09-10T22:57:33.635935+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:57:33.635935+0000","last_undegraded":"2024-09-10T22:57:33.635935+0000","last_fullsized":"2024-09-10T22:57:33.635935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488423+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:57:33.488423+0000","last_peered":"2024-09-10T22:57:33.488423+0000","last_clean":"2024-09-10T22:57:33.488423+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:57:33.488423+0000","last_undegraded":"2024-09-10T22:57:33.488423+0000","last_fullsized":"2024-09-10T22:57:33.488423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490516+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:57:33.490516+0000","last_peered":"2024-09-10T22:57:33.490516+0000","last_clean":"2024-09-10T22:57:33.490516+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:57:33.490516+0000","last_undegraded":"2024-09-10T22:57:33.490516+0000","last_fullsized":"2024-09-10T22:57:33.490516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485363+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:57:33.485363+0000","last_peered":"2024-09-10T22:57:33.485363+0000","last_clean":"2024-09-10T22:57:33.485363+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:57:33.485363+0000","last_undegraded":"2024-09-10T22:57:33.485363+0000","last_fullsized":"2024-09-10T22:57:33.485363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489898+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:57:33.489898+0000","last_peered":"2024-09-10T22:57:33.489898+0000","last_clean":"2024-09-10T22:57:33.489898+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:57:33.489898+0000","last_undegraded":"2024-09-10T22:57:33.489898+0000","last_fullsized":"2024-09-10T22:57:33.489898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484224+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:57:33.484224+0000","last_peered":"2024-09-10T22:57:33.484224+0000","last_clean":"2024-09-10T22:57:33.484224+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:57:33.484224+0000","last_undegraded":"2024-09-10T22:57:33.484224+0000","last_fullsized":"2024-09-10T22:57:33.484224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491907+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:57:33.491907+0000","last_peered":"2024-09-10T22:57:33.491907+0000","last_clean":"2024-09-10T22:57:33.491907+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:57:33.491907+0000","last_undegraded":"2024-09-10T22:57:33.491907+0000","last_fullsized":"2024-09-10T22:57:33.491907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491892+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:57:33.491892+0000","last_peered":"2024-09-10T22:57:33.491892+0000","last_clean":"2024-09-10T22:57:33.491892+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:57:33.491892+0000","last_undegraded":"2024-09-10T22:57:33.491892+0000","last_fullsized":"2024-09-10T22:57:33.491892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635943+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:57:33.635943+0000","last_peered":"2024-09-10T22:57:33.635943+0000","last_clean":"2024-09-10T22:57:33.635943+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:57:33.635943+0000","last_undegraded":"2024-09-10T22:57:33.635943+0000","last_fullsized":"2024-09-10T22:57:33.635943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934287+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:57:33.934287+0000","last_peered":"2024-09-10T22:57:33.934287+0000","last_clean":"2024-09-10T22:57:33.934287+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:57:33.934287+0000","last_undegraded":"2024-09-10T22:57:33.934287+0000","last_fullsized":"2024-09-10T22:57:33.934287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487645+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:57:33.487645+0000","last_peered":"2024-09-10T22:57:33.487645+0000","last_clean":"2024-09-10T22:57:33.487645+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:57:33.487645+0000","last_undegraded":"2024-09-10T22:57:33.487645+0000","last_fullsized":"2024-09-10T22:57:33.487645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490348+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:57:33.490348+0000","last_peered":"2024-09-10T22:57:33.490348+0000","last_clean":"2024-09-10T22:57:33.490348+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:57:33.490348+0000","last_undegraded":"2024-09-10T22:57:33.490348+0000","last_fullsized":"2024-09-10T22:57:33.490348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485308+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:57:33.485308+0000","last_peered":"2024-09-10T22:57:33.485308+0000","last_clean":"2024-09-10T22:57:33.485308+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:57:33.485308+0000","last_undegraded":"2024-09-10T22:57:33.485308+0000","last_fullsized":"2024-09-10T22:57:33.485308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487369+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:57:33.487369+0000","last_peered":"2024-09-10T22:57:33.487369+0000","last_clean":"2024-09-10T22:57:33.487369+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:57:33.487369+0000","last_undegraded":"2024-09-10T22:57:33.487369+0000","last_fullsized":"2024-09-10T22:57:33.487369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495990+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:57:33.495990+0000","last_peered":"2024-09-10T22:57:33.495990+0000","last_clean":"2024-09-10T22:57:33.495990+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:57:33.495990+0000","last_undegraded":"2024-09-10T22:57:33.495990+0000","last_fullsized":"2024-09-10T22:57:33.495990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494336+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:57:33.494336+0000","last_peered":"2024-09-10T22:57:33.494336+0000","last_clean":"2024-09-10T22:57:33.494336+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:57:33.494336+0000","last_undegraded":"2024-09-10T22:57:33.494336+0000","last_fullsized":"2024-09-10T22:57:33.494336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933856+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:57:33.933856+0000","last_peered":"2024-09-10T22:57:33.933856+0000","last_clean":"2024-09-10T22:57:33.933856+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:57:33.933856+0000","last_undegraded":"2024-09-10T22:57:33.933856+0000","last_fullsized":"2024-09-10T22:57:33.933856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492277+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:57:33.492277+0000","last_peered":"2024-09-10T22:57:33.492277+0000","last_clean":"2024-09-10T22:57:33.492277+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:57:33.492277+0000","last_undegraded":"2024-09-10T22:57:33.492277+0000","last_fullsized":"2024-09-10T22:57:33.492277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492372+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:57:33.492372+0000","last_peered":"2024-09-10T22:57:33.492372+0000","last_clean":"2024-09-10T22:57:33.492372+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:57:33.492372+0000","last_undegraded":"2024-09-10T22:57:33.492372+0000","last_fullsized":"2024-09-10T22:57:33.492372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489777+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:57:33.489777+0000","last_peered":"2024-09-10T22:57:33.489777+0000","last_clean":"2024-09-10T22:57:33.489777+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:57:33.489777+0000","last_undegraded":"2024-09-10T22:57:33.489777+0000","last_fullsized":"2024-09-10T22:57:33.489777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":220,"num_read_kb":183,"num_write":168,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612640,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":499051,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1148279,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":503271,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":498983,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397559,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":1095004,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":494351,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":1090592,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":499113,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117071,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":494545,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378711,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":556949,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542236,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":557346,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":495739,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:57:47.864 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:57:47.865 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T22:57:47.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T22:57:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:47 smithi145 ceph-mon[34397]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:47 smithi145 ceph-mon[34397]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:47 smithi092 ceph-mon[34223]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:47 smithi092 ceph-mon[34223]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:57:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 -- 172.21.15.67:0/3983777308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04108620 msgr2=0x7f2b04108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 --2- 172.21.15.67:0/3983777308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04108620 0x7f2b04108a60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009900 tx=0x7f2af002a410 comp rx=0 tx=0).stop 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 -- 172.21.15.67:0/3983777308 shutdown_connections 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 --2- 172.21.15.67:0/3983777308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 0x7f2b0410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 --2- 172.21.15.67:0/3983777308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04108620 0x7f2b04108a60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 --2- 172.21.15.67:0/3983777308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04102c10 0x7f2b04102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 -- 172.21.15.67:0/3983777308 >> 172.21.15.67:0/3983777308 conn(0x7f2b040fe660 msgr2=0x7f2b04100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 -- 172.21.15.67:0/3983777308 shutdown_connections 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.869+0000 7f2b03e00640 1 -- 172.21.15.67:0/3983777308 wait complete. 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.870+0000 7f2b03e00640 1 Processor -- start 2024-09-10T22:57:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.870+0000 7f2b03e00640 1 -- start start 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.870+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04108620 0x7f2b0419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 0x7f2b041a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04112b10 con 0x7f2b04102c10 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04112990 con 0x7f2b04108620 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04112c90 con 0x7f2b04108fa0 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39154/0 (socket says 172.21.15.67:39154) 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 -- 172.21.15.67:0/2882311398 learned_addr learned my addr 172.21.15.67:0/2882311398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b03400640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 0x7f2b041a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02000640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04108620 0x7f2b0419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 -- 172.21.15.67:0/2882311398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 msgr2=0x7f2b041a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 0x7f2b041a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 -- 172.21.15.67:0/2882311398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04108620 msgr2=0x7f2b0419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04108620 0x7f2b0419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.871+0000 7f2b02a00640 1 -- 172.21.15.67:0/2882311398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b041a11c0 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b02a00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d770 tx=0x7f2af400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4017070 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b041a14b0 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2af40054b0 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041a7b70 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.872+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4020060 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.873+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b04102ff0 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.876+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f2af4004c00 con 0x7f2b04102c10 2024-09-10T22:57:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.877+0000 7f2b08a00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 0x7f2ae80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.877+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f2af40e6080 con 0x7f2b04102c10 2024-09-10T22:57:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.878+0000 7f2b02000640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 0x7f2ae80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.878+0000 7f2b02000640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 0x7f2ae80b7920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2af00057c0 tx=0x7f2af00058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:48.879+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2af40ac310 con 0x7f2b04102c10 2024-09-10T22:57:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.147+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2b0419c240 con 0x7f2b04102c10 2024-09-10T22:57:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.150+0000 7f2b08a00640 1 -- 172.21.15.67:0/2882311398 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f2b0419c240 con 0x7f2b04102c10 2024-09-10T22:57:49.153 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-09-10T22:57:49.153 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDtzuBmKFTiCBAApof0DL3wnSWwo2SFRUQdFw== 2024-09-10T22:57:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 msgr2=0x7f2ae80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 0x7f2ae80b7920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2af00057c0 tx=0x7f2af00058d0 comp rx=0 tx=0).stop 2024-09-10T22:57:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 msgr2=0x7f2b0419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2af400d770 tx=0x7f2af400dc30 comp rx=0 tx=0).stop 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 shutdown_connections 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2ae80b5460 0x7f2ae80b7920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b04108fa0 0x7f2b041a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b04108620 0x7f2b0419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 --2- 172.21.15.67:0/2882311398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b04102c10 0x7f2b0419b110 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.152+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 >> 172.21.15.67:0/2882311398 conn(0x7f2b040fe660 msgr2=0x7f2b04104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.153+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 shutdown_connections 2024-09-10T22:57:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:49.153+0000 7f2b03e00640 1 -- 172.21.15.67:0/2882311398 wait complete. 2024-09-10T22:57:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:49 smithi067 ceph-mon[30939]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:57:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:57:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:49 smithi145 ceph-mon[34397]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:57:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:57:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:49 smithi092 ceph-mon[34223]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T22:57:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:57:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2882311398' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:57:50.126 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T22:57:50.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T22:57:50.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T22:57:50.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T22:57:50.627 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 -- 172.21.15.67:0/1129094824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56441137e0 msgr2=0x7f5644115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 --2- 172.21.15.67:0/1129094824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56441137e0 0x7f5644115bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f563c009ca0 tx=0x7f563c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 -- 172.21.15.67:0/1129094824 shutdown_connections 2024-09-10T22:57:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 --2- 172.21.15.67:0/1129094824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56441137e0 0x7f5644115bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 --2- 172.21.15.67:0/1129094824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f56441132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 --2- 172.21.15.67:0/1129094824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56440776d0 0x7f5644077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 -- 172.21.15.67:0/1129094824 >> 172.21.15.67:0/1129094824 conn(0x7f5644100840 msgr2=0x7f5644102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 -- 172.21.15.67:0/1129094824 shutdown_connections 2024-09-10T22:57:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.240+0000 7f5643200640 1 -- 172.21.15.67:0/1129094824 wait complete. 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 Processor -- start 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 -- start start 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f564419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56441137e0 0x7f56441a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56441190b0 con 0x7f56440776d0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5644118f30 con 0x7f5644077ff0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.241+0000 7f5643200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5644119230 con 0x7f56441137e0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39178/0 (socket says 172.21.15.67:39178) 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 -- 172.21.15.67:0/2169921350 learned_addr learned my addr 172.21.15.67:0/2169921350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5639400640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f564419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5642800640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56441137e0 0x7f56441a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 -- 172.21.15.67:0/2169921350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56441137e0 msgr2=0x7f56441a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56441137e0 0x7f56441a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 -- 172.21.15.67:0/2169921350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 msgr2=0x7f564419fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f564419fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5641e00640 1 -- 172.21.15.67:0/2169921350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441a5590 con 0x7f56440776d0 2024-09-10T22:57:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.242+0000 7f5639400640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f564419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f5641e00640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f563400d950 tx=0x7f563400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634017030 con 0x7f56440776d0 2024-09-10T22:57:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5634004020 con 0x7f56440776d0 2024-09-10T22:57:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340055d0 con 0x7f56440776d0 2024-09-10T22:57:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56441a5880 con 0x7f56440776d0 2024-09-10T22:57:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.243+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441ac110 con 0x7f56440776d0 2024-09-10T22:57:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.244+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5644078be0 con 0x7f56440776d0 2024-09-10T22:57:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.247+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f5634009a30 con 0x7f56440776d0 2024-09-10T22:57:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.248+0000 7f563b400640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 0x7f56280b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.248+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f56340e61e0 con 0x7f56440776d0 2024-09-10T22:57:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.248+0000 7f5639400640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 0x7f56280b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.249+0000 7f5639400640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 0x7f56280b79a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56441034c0 tx=0x7f562c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.250+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f56340acb40 con 0x7f56440776d0 2024-09-10T22:57:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.449+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f5644077ab0 con 0x7f56440776d0 2024-09-10T22:57:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.449+0000 7f563b400640 1 -- 172.21.15.67:0/2169921350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v52) ==== 86+0+133 (secure 0 0 0) 0x7f5644077ab0 con 0x7f56440776d0 2024-09-10T22:57:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 msgr2=0x7f56280b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 0x7f56280b79a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56441034c0 tx=0x7f562c005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 msgr2=0x7f564419f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f563400d950 tx=0x7f563400de10 comp rx=0 tx=0).stop 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 shutdown_connections 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56280b54e0 0x7f56280b79a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56441137e0 0x7f56441a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5644077ff0 0x7f564419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 --2- 172.21.15.67:0/2169921350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56440776d0 0x7f564419f4e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.452+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 >> 172.21.15.67:0/2169921350 conn(0x7f5644100840 msgr2=0x7f5644102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.453+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 shutdown_connections 2024-09-10T22:57:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:51.453+0000 7f5643200640 1 -- 172.21.15.67:0/2169921350 wait complete. 2024-09-10T22:57:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:51 smithi067 ceph-mon[30939]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:51 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2169921350' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:57:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:51 smithi145 ceph-mon[34397]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:51 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2169921350' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:57:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:51 smithi092 ceph-mon[34223]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:51 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2169921350' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:57:52.241 INFO:teuthology.orchestra.run.smithi067.stdout:[{"version":52,"timestamp":"2024-09-10T22:56:07.759807+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-10T22:57:52.241 INFO:tasks.ceph_manager:config epoch is 52 2024-09-10T22:57:52.242 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:57:52.242 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:57:52.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T22:57:52.624 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:53.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 -- 172.21.15.67:0/3800492121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 msgr2=0x7f7334077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:53.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 --2- 172.21.15.67:0/3800492121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f7334077ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7324009900 tx=0x7f732402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 -- 172.21.15.67:0/3800492121 shutdown_connections 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 --2- 172.21.15.67:0/3800492121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f7334115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 --2- 172.21.15.67:0/3800492121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 0x7f73341132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 --2- 172.21.15.67:0/3800492121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f7334077ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 -- 172.21.15.67:0/3800492121 >> 172.21.15.67:0/3800492121 conn(0x7f7334100840 msgr2=0x7f7334102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 -- 172.21.15.67:0/3800492121 shutdown_connections 2024-09-10T22:57:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.210+0000 7f7333200640 1 -- 172.21.15.67:0/3800492121 wait complete. 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.211+0000 7f7333200640 1 Processor -- start 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.211+0000 7f7333200640 1 -- start start 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 0x7f733419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f73341a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73341190d0 con 0x7f73340776d0 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7334118f50 con 0x7f7334077ff0 2024-09-10T22:57:53.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7333200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7334119250 con 0x7f73341137e0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7331e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7331e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39192/0 (socket says 172.21.15.67:39192) 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7331e00640 1 -- 172.21.15.67:0/3265520203 learned_addr learned my addr 172.21.15.67:0/3265520203 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.212+0000 7f7329400640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 0x7f733419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7332800640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f73341a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 -- 172.21.15.67:0/3265520203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 msgr2=0x7f73341a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f73341a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 -- 172.21.15.67:0/3265520203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 msgr2=0x7f733419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 0x7f733419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 -- 172.21.15.67:0/3265520203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73341a5510 con 0x7f73340776d0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7332800640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f73341a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7331e00640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f732402a910 tx=0x7f7324003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:53.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732403f070 con 0x7f73340776d0 2024-09-10T22:57:53.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7324003e40 con 0x7f73340776d0 2024-09-10T22:57:53.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324035030 con 0x7f73340776d0 2024-09-10T22:57:53.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.213+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a57a0 con 0x7f73340776d0 2024-09-10T22:57:53.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.214+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341abec0 con 0x7f73340776d0 2024-09-10T22:57:53.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.215+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334077ab0 con 0x7f73340776d0 2024-09-10T22:57:53.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.217+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f7324031400 con 0x7f73340776d0 2024-09-10T22:57:53.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.218+0000 7f732b400640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 0x7f73180b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:53.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.219+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f7324106ca0 con 0x7f73340776d0 2024-09-10T22:57:53.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.219+0000 7f7329400640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 0x7f73180b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:53.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.219+0000 7f7329400640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 0x7f73180b79a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f733407a0f0 tx=0x7f731c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:53.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.222+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f73240cd580 con 0x7f73340776d0 2024-09-10T22:57:53.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.467+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f73341ac240 con 0x7f73340776d0 2024-09-10T22:57:53.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.476+0000 7f732b400640 1 -- 172.21.15.67:0/3265520203 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+294230 (secure 0 0 0) 0x7f73240cce80 con 0x7f73340776d0 2024-09-10T22:57:53.482 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 msgr2=0x7f73180b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 0x7f73180b79a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f733407a0f0 tx=0x7f731c00b390 comp rx=0 tx=0).stop 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 msgr2=0x7f733419f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f732402a910 tx=0x7f7324003c20 comp rx=0 tx=0).stop 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 shutdown_connections 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73180b54e0 0x7f73180b79a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73341137e0 0x7f73341a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7334077ff0 0x7f733419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 --2- 172.21.15.67:0/3265520203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73340776d0 0x7f733419f460 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 >> 172.21.15.67:0/3265520203 conn(0x7f7334100840 msgr2=0x7f7334102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.480+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 shutdown_connections 2024-09-10T22:57:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:53.481+0000 7f7333200640 1 -- 172.21.15.67:0/3265520203 wait complete. 2024-09-10T22:57:53.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:53 smithi067 ceph-mon[30939]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:53.800 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:53 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3265520203' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:57:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:53 smithi145 ceph-mon[34397]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:53 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3265520203' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:57:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:53 smithi092 ceph-mon[34223]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T22:57:54.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:53 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3265520203' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:57:54.250 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1232465191},{"type":"v1","addr":"172.21.15.67:6801","nonce":1232465191}]},"active_addr":"172.21.15.67:6801/1232465191","active_change":"2024-09-10T22:48:31.958262+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24169,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34163,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2947326901}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3158436818}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":910979027}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":633327851}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":4205746883}]}]} 2024-09-10T22:57:54.339 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T22:57:54.339 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T22:57:54.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:57:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 -- 172.21.15.67:0/449013755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 msgr2=0x7fc3ec10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 --2- 172.21.15.67:0/449013755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec10adf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009900 tx=0x7fc3d802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 -- 172.21.15.67:0/449013755 shutdown_connections 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 --2- 172.21.15.67:0/449013755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec10b330 0x7fc3ec111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 --2- 172.21.15.67:0/449013755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec10adf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 --2- 172.21.15.67:0/449013755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 -- 172.21.15.67:0/449013755 >> 172.21.15.67:0/449013755 conn(0x7fc3ec100a50 msgr2=0x7fc3ec102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.304+0000 7fc3f1600640 1 -- 172.21.15.67:0/449013755 shutdown_connections 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.305+0000 7fc3f1600640 1 -- 172.21.15.67:0/449013755 wait complete. 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 Processor -- start 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 -- start start 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec10b330 0x7fc3ec1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec114e30 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec114cb0 con 0x7fc3ec104fa0 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.306+0000 7fc3f1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec114fb0 con 0x7fc3ec10b330 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39214/0 (socket says 172.21.15.67:39214) 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 -- 172.21.15.67:0/3233178512 learned_addr learned my addr 172.21.15.67:0/3233178512 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3eaa00640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3eb400640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec10b330 0x7fc3ec1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 -- 172.21.15.67:0/3233178512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec10b330 msgr2=0x7fc3ec1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec10b330 0x7fc3ec1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 -- 172.21.15.67:0/3233178512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 msgr2=0x7fc3ec19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3ea000640 1 -- 172.21.15.67:0/3233178512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a1450 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.307+0000 7fc3eaa00640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3ea000640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc3d802a910 tx=0x7fc3d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d803f070 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d8003e40 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8035030 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a16e0 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.308+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a7e00 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.309+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec105380 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.311+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc3d8031400 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.313+0000 7fc3e3400640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 0x7fc3d00b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.313+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fc3d8106fb0 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.313+0000 7fc3eaa00640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 0x7fc3d00b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.314+0000 7fc3eaa00640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 0x7fc3d00b7900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00bfd0 tx=0x7fc3dc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:55.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.315+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc3d80cd240 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.512+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ec19c4d0 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.514+0000 7fc3e3400640 1 -- 172.21.15.67:0/3233178512 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24987 (secure 0 0 0) 0x7fc3d80d1c70 con 0x7fc3ec10a9b0 2024-09-10T22:57:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:55.515 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:57:33.473910+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:57:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.516+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 msgr2=0x7fc3d00b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.516+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 0x7fc3d00b7900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00bfd0 tx=0x7fc3dc00b390 comp rx=0 tx=0).stop 2024-09-10T22:57:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.516+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 msgr2=0x7fc3ec19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.516+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc3d802a910 tx=0x7fc3d8003c20 comp rx=0 tx=0).stop 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 shutdown_connections 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3d00b5440 0x7fc3d00b7900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3ec10b330 0x7fc3ec1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3ec10a9b0 0x7fc3ec19b8e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 --2- 172.21.15.67:0/3233178512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3ec104fa0 0x7fc3ec19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 >> 172.21.15.67:0/3233178512 conn(0x7fc3ec100a50 msgr2=0x7fc3ec102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 shutdown_connections 2024-09-10T22:57:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:55.517+0000 7fc3f1600640 1 -- 172.21.15.67:0/3233178512 wait complete. 2024-09-10T22:57:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:55 smithi067 ceph-mon[30939]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:57:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:55 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3233178512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:55 smithi145 ceph-mon[34397]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:57:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:55 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3233178512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:55 smithi092 ceph-mon[34223]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:57:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:55 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3233178512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:56.330 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T22:57:56.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:57:56.715 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:57.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 -- 172.21.15.67:0/3425027595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc10b180 msgr2=0x7fa8cc111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 --2- 172.21.15.67:0/3425027595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc10b180 0x7fa8cc111980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009ca0 tx=0x7fa8c402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 -- 172.21.15.67:0/3425027595 shutdown_connections 2024-09-10T22:57:57.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 --2- 172.21.15.67:0/3425027595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc10b180 0x7fa8cc111980 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 --2- 172.21.15.67:0/3425027595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10a800 0x7fa8cc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 --2- 172.21.15.67:0/3425027595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc104df0 0x7fa8cc1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 -- 172.21.15.67:0/3425027595 >> 172.21.15.67:0/3425027595 conn(0x7fa8cc100840 msgr2=0x7fa8cc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 -- 172.21.15.67:0/3425027595 shutdown_connections 2024-09-10T22:57:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.284+0000 7fa8d2200640 1 -- 172.21.15.67:0/3425027595 wait complete. 2024-09-10T22:57:57.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.285+0000 7fa8d2200640 1 Processor -- start 2024-09-10T22:57:57.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.285+0000 7fa8d2200640 1 -- start start 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.285+0000 7fa8d2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.285+0000 7fa8d2200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc10a800 0x7fa8cc19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 0x7fa8cc1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114d40 con 0x7fa8cc104df0 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d2200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114bc0 con 0x7fa8cc10a800 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d2200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114ec0 con 0x7fa8cc10b180 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39222/0 (socket says 172.21.15.67:39222) 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 -- 172.21.15.67:0/2962648932 learned_addr learned my addr 172.21.15.67:0/2962648932 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d1800640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 0x7fa8cc1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8c3e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc10a800 0x7fa8cc19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 -- 172.21.15.67:0/2962648932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 msgr2=0x7fa8cc1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 0x7fa8cc1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 -- 172.21.15.67:0/2962648932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc10a800 msgr2=0x7fa8cc19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc10a800 0x7fa8cc19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d0e00640 1 -- 172.21.15.67:0/2962648932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc1a1160 con 0x7fa8cc104df0 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.286+0000 7fa8d1800640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 0x7fa8cc1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8d0e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00d950 tx=0x7fa8bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc017070 con 0x7fa8cc104df0 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc009c30 con 0x7fa8cc104df0 2024-09-10T22:57:57.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1a1450 con 0x7fa8cc104df0 2024-09-10T22:57:57.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00f620 con 0x7fa8cc104df0 2024-09-10T22:57:57.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.287+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a7b10 con 0x7fa8cc104df0 2024-09-10T22:57:57.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.288+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc1051d0 con 0x7fa8cc104df0 2024-09-10T22:57:57.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.290+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fa8bc010460 con 0x7fa8cc104df0 2024-09-10T22:57:57.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.292+0000 7fa8ca000640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 0x7fa8b00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.292+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fa8bc0e5f50 con 0x7fa8cc104df0 2024-09-10T22:57:57.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.292+0000 7fa8c3e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 0x7fa8b00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.293+0000 7fa8c3e00640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 0x7fa8b00b7920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005fd0 tx=0x7fa8b4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.295+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8bc0ac830 con 0x7fa8cc104df0 2024-09-10T22:57:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.493+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8cc1a7e50 con 0x7fa8cc104df0 2024-09-10T22:57:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.495+0000 7fa8ca000640 1 -- 172.21.15.67:0/2962648932 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24987 (secure 0 0 0) 0x7fa8bc009de0 con 0x7fa8cc104df0 2024-09-10T22:57:57.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:57:57.497 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:57:33.473910+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:57:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 msgr2=0x7fa8b00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 0x7fa8b00b7920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005fd0 tx=0x7fa8b4005850 comp rx=0 tx=0).stop 2024-09-10T22:57:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 msgr2=0x7fa8cc19b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00d950 tx=0x7fa8bc00de10 comp rx=0 tx=0).stop 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 shutdown_connections 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8b00b5460 0x7fa8b00b7920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8cc10b180 0x7fa8cc1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8cc10a800 0x7fa8cc19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 --2- 172.21.15.67:0/2962648932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8cc104df0 0x7fa8cc19b0b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 >> 172.21.15.67:0/2962648932 conn(0x7fa8cc100840 msgr2=0x7fa8cc102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 shutdown_connections 2024-09-10T22:57:57.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:57:57.498+0000 7fa8d2200640 1 -- 172.21.15.67:0/2962648932 wait complete. 2024-09-10T22:57:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:57 smithi067 ceph-mon[30939]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:57:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:57 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2962648932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:57 smithi145 ceph-mon[34397]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:57:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:57 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2962648932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:57 smithi092 ceph-mon[34223]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:57:58.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:57 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2962648932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:57:58.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T22:57:58.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T22:57:58.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T22:57:58.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T22:57:58.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T22:57:58.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T22:57:58.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T22:57:58.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T22:57:58.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T22:57:58.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T22:57:58.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T22:57:58.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T22:57:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:59.343 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:59.770 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:57:59 smithi067 ceph-mon[30939]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:57:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:57:59 smithi145 ceph-mon[34397]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:57:59 smithi092 ceph-mon[34223]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 -- 172.21.15.67:0/3122826814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075540 msgr2=0x7faa30075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 --2- 172.21.15.67:0/3122826814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075540 0x7faa30075920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faa20009900 tx=0x7faa2002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 -- 172.21.15.67:0/3122826814 shutdown_connections 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 --2- 172.21.15.67:0/3122826814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30113b20 0x7faa30115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 --2- 172.21.15.67:0/3122826814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30075e60 0x7faa301135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 --2- 172.21.15.67:0/3122826814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075540 0x7faa30075920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 -- 172.21.15.67:0/3122826814 >> 172.21.15.67:0/3122826814 conn(0x7faa300fe830 msgr2=0x7faa30100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 -- 172.21.15.67:0/3122826814 shutdown_connections 2024-09-10T22:58:00.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.246+0000 7faa37c00640 1 -- 172.21.15.67:0/3122826814 wait complete. 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 Processor -- start 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 -- start start 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30075540 0x7faa3019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 0x7faa3019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30113b20 0x7faa301a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa301192a0 con 0x7faa30075e60 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa30119120 con 0x7faa30113b20 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.247+0000 7faa37c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa30119420 con 0x7faa30075540 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 0x7faa3019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 0x7faa3019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49490/0 (socket says 172.21.15.67:49490) 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 -- 172.21.15.67:0/946070613 learned_addr learned my addr 172.21.15.67:0/946070613 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:00.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa35c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30113b20 0x7faa301a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa35200640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30075540 0x7faa3019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 -- 172.21.15.67:0/946070613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30075540 msgr2=0x7faa3019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30075540 0x7faa3019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 -- 172.21.15.67:0/946070613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30113b20 msgr2=0x7faa301a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30113b20 0x7faa301a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301a5900 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa35200640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30075540 0x7faa3019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.248+0000 7faa27e00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 0x7faa3019fd90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9f0 tx=0x7faa1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.249+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1800cd80 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.249+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa180108c0 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.249+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa180187c0 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.249+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa301a5bf0 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.249+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa301ac2b0 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.250+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa30100ea0 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.252+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7faa18029050 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.253+0000 7faa2e000640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa140b53c0 0x7faa140b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7faa180e6b70 con 0x7faa30075e60 2024-09-10T22:58:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa35200640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa140b53c0 0x7faa140b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa2e000640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] conn(0x7faa140c0680 0x7faa140c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa140c31f0 con 0x7faa140c0680 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7faa180e6f20 con 0x7faa30075e60 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.254+0000 7faa35c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] conn(0x7faa140c0680 0x7faa140c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.255+0000 7faa35200640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa140b53c0 0x7faa140b7880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa20006010 tx=0x7faa20005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.255+0000 7faa35c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] conn(0x7faa140c0680 0x7faa140c2b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.256+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== osd.0 v2:172.21.15.67:6802/3409065493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa140c31f0 con 0x7faa140c0680 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.270+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 --> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa30076a50 con 0x7faa140c0680 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa2e000640 1 -- 172.21.15.67:0/946070613 <== osd.0 v2:172.21.15.67:6802/3409065493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa30076a50 con 0x7faa140c0680 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] conn(0x7faa140c0680 msgr2=0x7faa140c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] conn(0x7faa140c0680 0x7faa140c2b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa140b53c0 msgr2=0x7faa140b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa140b53c0 0x7faa140b7880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa20006010 tx=0x7faa20005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 msgr2=0x7faa3019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 --2- 172.21.15.67:0/946070613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa30075e60 0x7faa3019fd90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9f0 tx=0x7faa1800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa35c00640 1 -- 172.21.15.67:0/946070613 reap_dead start 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 shutdown_connections 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 >> 172.21.15.67:0/946070613 conn(0x7faa300fe830 msgr2=0x7faa300ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.271+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 shutdown_connections 2024-09-10T22:58:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:00.272+0000 7faa37c00640 1 -- 172.21.15.67:0/946070613 wait complete. 2024-09-10T22:58:01.088 INFO:teuthology.orchestra.run.smithi067.stdout:38654705766 2024-09-10T22:58:01.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T22:58:01.266 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 -- 172.21.15.67:0/3287872052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f4102dc0 msgr2=0x7fc9f41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 --2- 172.21.15.67:0/3287872052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f4102dc0 0x7fc9f41031a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009ca0 tx=0x7fc9e402a8b0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 -- 172.21.15.67:0/3287872052 shutdown_connections 2024-09-10T22:58:01.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 --2- 172.21.15.67:0/3287872052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9f4109150 0x7fc9f410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 --2- 172.21.15.67:0/3287872052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f41087d0 0x7fc9f4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 --2- 172.21.15.67:0/3287872052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f4102dc0 0x7fc9f41031a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.373+0000 7fc9fa600640 1 -- 172.21.15.67:0/3287872052 >> 172.21.15.67:0/3287872052 conn(0x7fc9f40fe830 msgr2=0x7fc9f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.374+0000 7fc9fa600640 1 -- 172.21.15.67:0/3287872052 shutdown_connections 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.374+0000 7fc9fa600640 1 -- 172.21.15.67:0/3287872052 wait complete. 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.375+0000 7fc9fa600640 1 Processor -- start 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.375+0000 7fc9fa600640 1 -- start start 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.375+0000 7fc9fa600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4102dc0 0x7fc9f41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9fa600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 0x7fc9f4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9fa600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9f4109150 0x7fc9f419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9fa600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4112d10 con 0x7fc9f41087d0 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9fa600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4112b90 con 0x7fc9f4102dc0 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9fa600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4112e90 con 0x7fc9f4109150 2024-09-10T22:58:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 0x7fc9f4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 0x7fc9f4199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49510/0 (socket says 172.21.15.67:49510) 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 -- 172.21.15.67:0/251709630 learned_addr learned my addr 172.21.15.67:0/251709630 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9f3e00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9f4109150 0x7fc9f419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9f3400640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4102dc0 0x7fc9f41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 -- 172.21.15.67:0/251709630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9f4109150 msgr2=0x7fc9f419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9f4109150 0x7fc9f419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 -- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4102dc0 msgr2=0x7fc9f41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4102dc0 0x7fc9f41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.376+0000 7fc9eaa00640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f419f280 con 0x7fc9f41087d0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9eaa00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 0x7fc9f4199710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017030 con 0x7fc9f41087d0 2024-09-10T22:58:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9dc004020 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0055d0 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f419f570 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.377+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f41a5c30 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.378+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9f40638e0 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.381+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc9dc009a30 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f0c00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc9d80b5440 0x7fc9d80b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fc9dc0e63b0 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f3400640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc9d80b5440 0x7fc9d80b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f0c00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] conn(0x7fc9d80c0700 0x7fc9d80c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9dc003e50 con 0x7fc9d80c0700 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.382+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc0e6760 con 0x7fc9f41087d0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.383+0000 7fc9f3e00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] conn(0x7fc9d80c0700 0x7fc9d80c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.383+0000 7fc9f3400640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc9d80b5440 0x7fc9d80b7900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000f80 tx=0x7fc9e40099e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.383+0000 7fc9f3e00640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] conn(0x7fc9d80c0700 0x7fc9d80c2ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.385+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== osd.7 v2:172.21.15.92:6816/3770526101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9dc003e50 con 0x7fc9d80c0700 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.399+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 --> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9f4063db0 con 0x7fc9d80c0700 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.399+0000 7fc9f0c00640 1 -- 172.21.15.67:0/251709630 <== osd.7 v2:172.21.15.92:6816/3770526101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9f4063db0 con 0x7fc9d80c0700 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] conn(0x7fc9d80c0700 msgr2=0x7fc9d80c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] conn(0x7fc9d80c0700 0x7fc9d80c2ba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc9d80b5440 msgr2=0x7fc9d80b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc9d80b5440 0x7fc9d80b7900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000f80 tx=0x7fc9e40099e0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 msgr2=0x7fc9f4199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 --2- 172.21.15.67:0/251709630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9f41087d0 0x7fc9f4199710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).stop 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9f3e00640 1 -- 172.21.15.67:0/251709630 reap_dead start 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 shutdown_connections 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 >> 172.21.15.67:0/251709630 conn(0x7fc9f40fe830 msgr2=0x7fc9f41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 shutdown_connections 2024-09-10T22:58:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:01.400+0000 7fc9fa600640 1 -- 172.21.15.67:0/251709630 wait complete. 2024-09-10T22:58:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.708 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.710 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.710 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.716 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:01 smithi067 ceph-mon[30939]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:01 smithi145 ceph-mon[34397]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:01 smithi092 ceph-mon[34223]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:02.244 INFO:teuthology.orchestra.run.smithi067.stdout:210453397563 2024-09-10T22:58:02.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 -- 172.21.15.67:0/1827750034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff854103b10 msgr2=0x7ff854103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 --2- 172.21.15.67:0/1827750034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff854103b10 0x7ff854103ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff848009900 tx=0x7ff84802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 -- 172.21.15.67:0/1827750034 shutdown_connections 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 --2- 172.21.15.67:0/1827750034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff85410cf70 0x7ff85410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 --2- 172.21.15.67:0/1827750034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff854104430 0x7ff85410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 --2- 172.21.15.67:0/1827750034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff854103b10 0x7ff854103ef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 -- 172.21.15.67:0/1827750034 >> 172.21.15.67:0/1827750034 conn(0x7ff8540fd5b0 msgr2=0x7ff8540ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 -- 172.21.15.67:0/1827750034 shutdown_connections 2024-09-10T22:58:03.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.034+0000 7ff85a200640 1 -- 172.21.15.67:0/1827750034 wait complete. 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 Processor -- start 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 -- start start 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff854103b10 0x7ff8541017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff854104430 0x7ff854101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 0x7ff854102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff854112a90 con 0x7ff85410cf70 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff854112910 con 0x7ff854103b10 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.035+0000 7ff85a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff854112c10 con 0x7ff854104430 2024-09-10T22:58:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 0x7ff854102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 0x7ff854102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49522/0 (socket says 172.21.15.67:49522) 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 -- 172.21.15.67:0/93019720 learned_addr learned my addr 172.21.15.67:0/93019720 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff858e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff854103b10 0x7ff8541017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff853e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff854104430 0x7ff854101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 -- 172.21.15.67:0/93019720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff854104430 msgr2=0x7ff854101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff854104430 0x7ff854101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 -- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff854103b10 msgr2=0x7ff8541017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff854103b10 0x7ff8541017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.036+0000 7ff859800640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8541a7340 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff859800640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 0x7ff854102240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00eaa0 tx=0x7ff84c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00cdb0 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff84c005020 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8541a7630 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0116d0 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.037+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8541a7bc0 con 0x7ff85410cf70 2024-09-10T22:58:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.038+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8540ff510 con 0x7ff85410cf70 2024-09-10T22:58:03.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.041+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7ff84c00f040 con 0x7ff85410cf70 2024-09-10T22:58:03.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.042+0000 7ff851600640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff83c0b54e0 0x7ff83c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.042+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7ff84c0e6420 con 0x7ff85410cf70 2024-09-10T22:58:03.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.042+0000 7ff858e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff83c0b54e0 0x7ff83c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.043+0000 7ff851600640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] conn(0x7ff83c0c07a0 0x7ff83c0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.043+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff84c014ec0 con 0x7ff83c0c07a0 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.043+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff84c0e67d0 con 0x7ff85410cf70 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.043+0000 7ff853e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] conn(0x7ff83c0c07a0 0x7ff83c0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.043+0000 7ff858e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff83c0b54e0 0x7ff83c0b79a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8480057c0 tx=0x7ff8480058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.044+0000 7ff853e00640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] conn(0x7ff83c0c07a0 0x7ff83c0c2c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.045+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== osd.1 v2:172.21.15.92:6800/1917308558 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff84c014ec0 con 0x7ff83c0c07a0 2024-09-10T22:58:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.059+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 --> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff85406b1b0 con 0x7ff83c0c07a0 2024-09-10T22:58:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff851600640 1 -- 172.21.15.67:0/93019720 <== osd.1 v2:172.21.15.92:6800/1917308558 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff85406b1b0 con 0x7ff83c0c07a0 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] conn(0x7ff83c0c07a0 msgr2=0x7ff83c0c2c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] conn(0x7ff83c0c07a0 0x7ff83c0c2c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff83c0b54e0 msgr2=0x7ff83c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff83c0b54e0 0x7ff83c0b79a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8480057c0 tx=0x7ff8480058d0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 msgr2=0x7ff854102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff85a200640 1 --2- 172.21.15.67:0/93019720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff85410cf70 0x7ff854102240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00eaa0 tx=0x7ff84c00ef60 comp rx=0 tx=0).stop 2024-09-10T22:58:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.060+0000 7ff859800640 1 -- 172.21.15.67:0/93019720 reap_dead start 2024-09-10T22:58:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.061+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 shutdown_connections 2024-09-10T22:58:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.061+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 >> 172.21.15.67:0/93019720 conn(0x7ff8540fd5b0 msgr2=0x7ff85410f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.061+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 shutdown_connections 2024-09-10T22:58:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.061+0000 7ff85a200640 1 -- 172.21.15.67:0/93019720 wait complete. 2024-09-10T22:58:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 -- 172.21.15.67:0/786323866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb881087b0 msgr2=0x7ffb88108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 --2- 172.21.15.67:0/786323866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb881087b0 0x7ffb88108bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009900 tx=0x7ffb7402a410 comp rx=0 tx=0).stop 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 -- 172.21.15.67:0/786323866 shutdown_connections 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 --2- 172.21.15.67:0/786323866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88109130 0x7ffb8810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 --2- 172.21.15.67:0/786323866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb881087b0 0x7ffb88108bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 --2- 172.21.15.67:0/786323866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb88102da0 0x7ffb88103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 -- 172.21.15.67:0/786323866 >> 172.21.15.67:0/786323866 conn(0x7ffb880fe810 msgr2=0x7ffb88100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.502+0000 7ffb8d000640 1 -- 172.21.15.67:0/786323866 shutdown_connections 2024-09-10T22:58:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.503+0000 7ffb8d000640 1 -- 172.21.15.67:0/786323866 wait complete. 2024-09-10T22:58:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 Processor -- start 2024-09-10T22:58:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 -- start start 2024-09-10T22:58:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 0x7ffb881991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb881087b0 0x7ffb88199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb88109130 0x7ffb8819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88112cb0 con 0x7ffb88109130 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88112b30 con 0x7ffb881087b0 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.504+0000 7ffb8d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88112e30 con 0x7ffb88102da0 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 0x7ffb881991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb881087b0 0x7ffb88199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 0x7ffb881991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:49674/0 (socket says 172.21.15.67:49674) 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 -- 172.21.15.67:0/3307338762 learned_addr learned my addr 172.21.15.67:0/3307338762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:03.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb86800640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb88109130 0x7ffb8819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 -- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb881087b0 msgr2=0x7ffb88199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb881087b0 0x7ffb88199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 -- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb88109130 msgr2=0x7ffb8819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb88109130 0x7ffb8819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.505+0000 7ffb85e00640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8819f270 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb85e00640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 0x7ffb881991c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9f0 tx=0x7ffb7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800cd80 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8819f560 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb780108c0 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb780187c0 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.506+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb881a5df0 con 0x7ffb88102da0 2024-09-10T22:58:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.507+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb880638e0 con 0x7ffb88102da0 2024-09-10T22:58:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.510+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7ffb78029050 con 0x7ffb88102da0 2024-09-10T22:58:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.511+0000 7ffb7ea00640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb6c0b5440 0x7ffb6c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.511+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7ffb780e6a10 con 0x7ffb88102da0 2024-09-10T22:58:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.511+0000 7ffb85400640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb6c0b5440 0x7ffb6c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.512+0000 7ffb7ea00640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] conn(0x7ffb6c0c0700 0x7ffb6c0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.512+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb78027e50 con 0x7ffb6c0c0700 2024-09-10T22:58:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.512+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ffb780e6dc0 con 0x7ffb88102da0 2024-09-10T22:58:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.512+0000 7ffb85400640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb6c0b5440 0x7ffb6c0b7900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb74006010 tx=0x7ffb74005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.512+0000 7ffb86800640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] conn(0x7ffb6c0c0700 0x7ffb6c0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.513+0000 7ffb86800640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] conn(0x7ffb6c0c0700 0x7ffb6c0c2ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.514+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== osd.4 v2:172.21.15.92:6808/1079057674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb78027e50 con 0x7ffb6c0c0700 2024-09-10T22:58:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.529+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 --> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb88063d10 con 0x7ffb6c0c0700 2024-09-10T22:58:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.529+0000 7ffb7ea00640 1 -- 172.21.15.67:0/3307338762 <== osd.4 v2:172.21.15.92:6808/1079057674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb88063d10 con 0x7ffb6c0c0700 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] conn(0x7ffb6c0c0700 msgr2=0x7ffb6c0c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] conn(0x7ffb6c0c0700 0x7ffb6c0c2ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb6c0b5440 msgr2=0x7ffb6c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb6c0b5440 0x7ffb6c0b7900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb74006010 tx=0x7ffb74005e80 comp rx=0 tx=0).stop 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 msgr2=0x7ffb881991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 --2- 172.21.15.67:0/3307338762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb88102da0 0x7ffb881991c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9f0 tx=0x7ffb7800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb86800640 1 -- 172.21.15.67:0/3307338762 reap_dead start 2024-09-10T22:58:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 shutdown_connections 2024-09-10T22:58:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 >> 172.21.15.67:0/3307338762 conn(0x7ffb880fe810 msgr2=0x7ffb88100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 shutdown_connections 2024-09-10T22:58:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:03.530+0000 7ffb8d000640 1 -- 172.21.15.67:0/3307338762 wait complete. 2024-09-10T22:58:03.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:03 smithi067 ceph-mon[30939]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:03 smithi145 ceph-mon[34397]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:03 smithi092 ceph-mon[34223]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 -- 172.21.15.67:0/181123759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c10b330 msgr2=0x7ff96c111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 --2- 172.21.15.67:0/181123759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c10b330 0x7ff96c111b30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff964009ca0 tx=0x7ff96402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 -- 172.21.15.67:0/181123759 shutdown_connections 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 --2- 172.21.15.67:0/181123759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c10b330 0x7ff96c111b30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 --2- 172.21.15.67:0/181123759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10a9b0 0x7ff96c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 --2- 172.21.15.67:0/181123759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c104fa0 0x7ff96c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 -- 172.21.15.67:0/181123759 >> 172.21.15.67:0/181123759 conn(0x7ff96c100a50 msgr2=0x7ff96c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 -- 172.21.15.67:0/181123759 shutdown_connections 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.512+0000 7ff974400640 1 -- 172.21.15.67:0/181123759 wait complete. 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 Processor -- start 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 -- start start 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 0x7ff96c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10a9b0 0x7ff96c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10b330 0x7ff96c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c114e60 con 0x7ff96c104fa0 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c114ce0 con 0x7ff96c10a9b0 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff974400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c114fe0 con 0x7ff96c10b330 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.513+0000 7ff971a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 0x7ff96c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 0x7ff96c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49566/0 (socket says 172.21.15.67:49566) 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 -- 172.21.15.67:0/1070185461 learned_addr learned my addr 172.21.15.67:0/1070185461 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff972400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10b330 0x7ff96c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971000640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10a9b0 0x7ff96c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 -- 172.21.15.67:0/1070185461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10b330 msgr2=0x7ff96c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10b330 0x7ff96c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 -- 172.21.15.67:0/1070185461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10a9b0 msgr2=0x7ff96c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff96c10a9b0 0x7ff96c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1a1400 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff972400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff96c10b330 0x7ff96c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.514+0000 7ff971a00640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 0x7ff96c19b350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.515+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000cd80 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.515+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9600108c0 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.515+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c1a16f0 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.515+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9600187c0 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.515+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a7db0 con 0x7ff96c104fa0 2024-09-10T22:58:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.516+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff96c0638e0 con 0x7ff96c104fa0 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.518+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7ff960029050 con 0x7ff96c104fa0 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.519+0000 7ff96a000640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9540b53c0 0x7ff9540b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7ff9600e6bc0 con 0x7ff96c104fa0 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff971000640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9540b53c0 0x7ff9540b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff96a000640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] conn(0x7ff9540c0680 0x7ff9540c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9540c31f0 con 0x7ff9540c0680 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff9600e7010 con 0x7ff96c104fa0 2024-09-10T22:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff972400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] conn(0x7ff9540c0680 0x7ff9540c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.520+0000 7ff971000640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9540b53c0 0x7ff9540b7880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1096c0 tx=0x7ff958005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.521+0000 7ff972400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] conn(0x7ff9540c0680 0x7ff9540c2b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.522+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== osd.3 v2:172.21.15.67:6810/3883722074 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9540c31f0 con 0x7ff9540c0680 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.536+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 --> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff96c063d60 con 0x7ff9540c0680 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.536+0000 7ff96a000640 1 -- 172.21.15.67:0/1070185461 <== osd.3 v2:172.21.15.67:6810/3883722074 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff96c063d60 con 0x7ff9540c0680 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] conn(0x7ff9540c0680 msgr2=0x7ff9540c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] conn(0x7ff9540c0680 0x7ff9540c2b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9540b53c0 msgr2=0x7ff9540b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9540b53c0 0x7ff9540b7880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1096c0 tx=0x7ff958005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 msgr2=0x7ff96c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 --2- 172.21.15.67:0/1070185461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff96c104fa0 0x7ff96c19b350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff972400640 1 -- 172.21.15.67:0/1070185461 reap_dead start 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 shutdown_connections 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 >> 172.21.15.67:0/1070185461 conn(0x7ff96c100a50 msgr2=0x7ff96c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 shutdown_connections 2024-09-10T22:58:04.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:04.537+0000 7ff974400640 1 -- 172.21.15.67:0/1070185461 wait complete. 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 -- 172.21.15.67:0/2175568935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc144075f40 msgr2=0x7fc14410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 --2- 172.21.15.67:0/2175568935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc144075f40 0x7fc14410ee10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc130009900 tx=0x7fc13002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 -- 172.21.15.67:0/2175568935 shutdown_connections 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 --2- 172.21.15.67:0/2175568935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc14410f3e0 0x7fc1441117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 --2- 172.21.15.67:0/2175568935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc144075f40 0x7fc14410ee10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 --2- 172.21.15.67:0/2175568935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc144075540 0x7fc144075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 -- 172.21.15.67:0/2175568935 >> 172.21.15.67:0/2175568935 conn(0x7fc1440fe830 msgr2=0x7fc144100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:05.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 -- 172.21.15.67:0/2175568935 shutdown_connections 2024-09-10T22:58:05.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.460+0000 7fc14c400640 1 -- 172.21.15.67:0/2175568935 wait complete. 2024-09-10T22:58:05.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 Processor -- start 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 -- start start 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc144075540 0x7fc14419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc144075f40 0x7fc14419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 0x7fc1441a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc144114ef0 con 0x7fc14410f3e0 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc144114d70 con 0x7fc144075f40 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.461+0000 7fc14c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc144115070 con 0x7fc144075540 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 0x7fc1441a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 0x7fc1441a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49594/0 (socket says 172.21.15.67:49594) 2024-09-10T22:58:05.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 -- 172.21.15.67:0/3653693106 learned_addr learned my addr 172.21.15.67:0/3653693106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc149a00640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc144075540 0x7fc14419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc149000640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc144075f40 0x7fc14419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 -- 172.21.15.67:0/3653693106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc144075540 msgr2=0x7fc14419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc144075540 0x7fc14419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 -- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc144075f40 msgr2=0x7fc14419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc144075f40 0x7fc14419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.462+0000 7fc14a400640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1441a14f0 con 0x7fc14410f3e0 2024-09-10T22:58:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc14a400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 0x7fc1441a0d20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00e9d0 tx=0x7fc13c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c004d20 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc13c005020 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1441a17e0 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0116d0 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.463+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1441a7ea0 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.464+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc144100ea0 con 0x7fc14410f3e0 2024-09-10T22:58:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.467+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc13c00f040 con 0x7fc14410f3e0 2024-09-10T22:58:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.468+0000 7fc142000640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc12c0b53c0 0x7fc12c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fc13c0e64b0 con 0x7fc14410f3e0 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc149a00640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc12c0b53c0 0x7fc12c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc142000640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] conn(0x7fc12c0c0680 0x7fc12c0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc13c014ec0 con 0x7fc12c0c0680 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc13c0e6900 con 0x7fc14410f3e0 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc149000640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] conn(0x7fc12c0c0680 0x7fc12c0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.469+0000 7fc149a00640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc12c0b53c0 0x7fc12c0b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc138009990 tx=0x7fc138009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:05.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.470+0000 7fc149000640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] conn(0x7fc12c0c0680 0x7fc12c0c2b20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:05.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.471+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== osd.10 v2:172.21.15.92:6824/1466235369 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc13c014ec0 con 0x7fc12c0c0680 2024-09-10T22:58:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.486+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 --> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc144076a50 con 0x7fc12c0c0680 2024-09-10T22:58:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.486+0000 7fc142000640 1 -- 172.21.15.67:0/3653693106 <== osd.10 v2:172.21.15.92:6824/1466235369 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc144076a50 con 0x7fc12c0c0680 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] conn(0x7fc12c0c0680 msgr2=0x7fc12c0c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] conn(0x7fc12c0c0680 0x7fc12c0c2b20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc12c0b53c0 msgr2=0x7fc12c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc12c0b53c0 0x7fc12c0b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc138009990 tx=0x7fc138009110 comp rx=0 tx=0).stop 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 msgr2=0x7fc1441a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 --2- 172.21.15.67:0/3653693106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14410f3e0 0x7fc1441a0d20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00e9d0 tx=0x7fc13c00edd0 comp rx=0 tx=0).stop 2024-09-10T22:58:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14a400640 1 -- 172.21.15.67:0/3653693106 reap_dead start 2024-09-10T22:58:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 shutdown_connections 2024-09-10T22:58:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 >> 172.21.15.67:0/3653693106 conn(0x7fc1440fe830 msgr2=0x7fc1440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 shutdown_connections 2024-09-10T22:58:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:05.487+0000 7fc14c400640 1 -- 172.21.15.67:0/3653693106 wait complete. 2024-09-10T22:58:05.605 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:05 smithi067 ceph-mon[30939]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:05 smithi145 ceph-mon[34397]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:05 smithi092 ceph-mon[34223]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:06.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 -- 172.21.15.67:0/3159141541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 msgr2=0x7fb6e410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 --2- 172.21.15.67:0/3159141541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e410ade0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009900 tx=0x7fb6d002a400 comp rx=0 tx=0).stop 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 -- 172.21.15.67:0/3159141541 shutdown_connections 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 --2- 172.21.15.67:0/3159141541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6e410b320 0x7fb6e4111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 --2- 172.21.15.67:0/3159141541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e410ade0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 --2- 172.21.15.67:0/3159141541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 0x7fb6e4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 -- 172.21.15.67:0/3159141541 >> 172.21.15.67:0/3159141541 conn(0x7fb6e4100a00 msgr2=0x7fb6e4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 -- 172.21.15.67:0/3159141541 shutdown_connections 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.678+0000 7fb6ec000640 1 -- 172.21.15.67:0/3159141541 wait complete. 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.679+0000 7fb6ec000640 1 Processor -- start 2024-09-10T22:58:06.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.679+0000 7fb6ec000640 1 -- start start 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 0x7fb6e419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6e410b320 0x7fb6e41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114e20 con 0x7fb6e410a9a0 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114ca0 con 0x7fb6e4104f90 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ec000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114fa0 con 0x7fb6e410b320 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6e8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6e8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49608/0 (socket says 172.21.15.67:49608) 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6e9600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 0x7fb6e419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6e8c00640 1 -- 172.21.15.67:0/1412575060 learned_addr learned my addr 172.21.15.67:0/1412575060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.680+0000 7fb6ea000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6e410b320 0x7fb6e41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 -- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6e410b320 msgr2=0x7fb6e41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6e410b320 0x7fb6e41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 -- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 msgr2=0x7fb6e419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41a1380 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e8c00640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6e410a9a0 0x7fb6e419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e9600640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 0x7fb6e419b2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800cce0 tx=0x7fb6d800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800f450 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e41a1670 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6d800fa80 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8018a00 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.681+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41a7d30 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.682+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6e40638e0 con 0x7fb6e4104f90 2024-09-10T22:58:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.685+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb6d8018ba0 con 0x7fb6e4104f90 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e2000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6cc0b53c0 0x7fb6cc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fb6d80e6820 con 0x7fb6e4104f90 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e8c00640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6cc0b53c0 0x7fb6cc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e2000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] conn(0x7fb6cc0c0680 0x7fb6cc0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6d8012ec0 con 0x7fb6cc0c0680 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.686+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb6d80e6bd0 con 0x7fb6e4104f90 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.687+0000 7fb6ea000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] conn(0x7fb6cc0c0680 0x7fb6cc0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.687+0000 7fb6e8c00640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6cc0b53c0 0x7fb6cc0b7880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00096c0 tx=0x7fb6d0005b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.688+0000 7fb6ea000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] conn(0x7fb6cc0c0680 0x7fb6cc0c2b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.689+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== osd.5 v2:172.21.15.145:6808/510004066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6d8012ec0 con 0x7fb6cc0c0680 2024-09-10T22:58:06.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.705+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 --> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6e4063ce0 con 0x7fb6cc0c0680 2024-09-10T22:58:06.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.706+0000 7fb6e2000640 1 -- 172.21.15.67:0/1412575060 <== osd.5 v2:172.21.15.145:6808/510004066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb6e4063ce0 con 0x7fb6cc0c0680 2024-09-10T22:58:06.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] conn(0x7fb6cc0c0680 msgr2=0x7fb6cc0c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] conn(0x7fb6cc0c0680 0x7fb6cc0c2b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6cc0b53c0 msgr2=0x7fb6cc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6cc0b53c0 0x7fb6cc0b7880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00096c0 tx=0x7fb6d0005b50 comp rx=0 tx=0).stop 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 msgr2=0x7fb6e419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 --2- 172.21.15.67:0/1412575060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e4104f90 0x7fb6e419b2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800cce0 tx=0x7fb6d800a620 comp rx=0 tx=0).stop 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ea000640 1 -- 172.21.15.67:0/1412575060 reap_dead start 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 shutdown_connections 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 >> 172.21.15.67:0/1412575060 conn(0x7fb6e4100a00 msgr2=0x7fb6e4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 shutdown_connections 2024-09-10T22:58:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.707+0000 7fb6ec000640 1 -- 172.21.15.67:0/1412575060 wait complete. 2024-09-10T22:58:07.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 -- 172.21.15.67:0/2442094674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb044075540 msgr2=0x7fb044075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 --2- 172.21.15.67:0/2442094674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb044075540 0x7fb044075920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb034009900 tx=0x7fb03402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 -- 172.21.15.67:0/2442094674 shutdown_connections 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 --2- 172.21.15.67:0/2442094674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04410f3e0 0x7fb0441117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 --2- 172.21.15.67:0/2442094674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044075f40 0x7fb04410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 --2- 172.21.15.67:0/2442094674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb044075540 0x7fb044075920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 -- 172.21.15.67:0/2442094674 >> 172.21.15.67:0/2442094674 conn(0x7fb0440fe830 msgr2=0x7fb044100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 -- 172.21.15.67:0/2442094674 shutdown_connections 2024-09-10T22:58:07.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.256+0000 7fb049e00640 1 -- 172.21.15.67:0/2442094674 wait complete. 2024-09-10T22:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.257+0000 7fb049e00640 1 Processor -- start 2024-09-10T22:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 -- start start 2024-09-10T22:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044075540 0x7fb04419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044075f40 0x7fb04419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 0x7fb0441a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb044114ea0 con 0x7fb04410f3e0 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb044114d20 con 0x7fb044075f40 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb049e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb044115020 con 0x7fb044075540 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb043400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 0x7fb0441a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb043400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 0x7fb0441a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49644/0 (socket says 172.21.15.67:49644) 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb043400640 1 -- 172.21.15.67:0/3627759981 learned_addr learned my addr 172.21.15.67:0/3627759981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb042a00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044075540 0x7fb04419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.258+0000 7fb03be00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044075f40 0x7fb04419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 -- 172.21.15.67:0/3627759981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044075540 msgr2=0x7fb04419b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb044075540 0x7fb04419b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 -- 172.21.15.67:0/3627759981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044075f40 msgr2=0x7fb04419b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044075f40 0x7fb04419b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0441a1560 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb043400640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 0x7fb0441a0d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9d0 tx=0x7fb03c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00cd80 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.259+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb03c005020 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.260+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441a1850 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.260+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0116d0 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.260+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a7f10 con 0x7fb04410f3e0 2024-09-10T22:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.261+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0440638e0 con 0x7fb04410f3e0 2024-09-10T22:58:07.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.263+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb03c00f040 con 0x7fb04410f3e0 2024-09-10T22:58:07.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.264+0000 7fb048a00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5440 0x7fb0280b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fb03c0e6430 con 0x7fb04410f3e0 2024-09-10T22:58:07.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb042a00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5440 0x7fb0280b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb048a00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] conn(0x7fb0280c0700 0x7fb0280c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb03c014ec0 con 0x7fb0280c0700 2024-09-10T22:58:07.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb03c0e67e0 con 0x7fb04410f3e0 2024-09-10T22:58:07.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb03be00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] conn(0x7fb0280c0700 0x7fb0280c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.265+0000 7fb042a00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5440 0x7fb0280b7900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb034006010 tx=0x7fb034005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.266+0000 7fb03be00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] conn(0x7fb0280c0700 0x7fb0280c2ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.267+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== osd.6 v2:172.21.15.67:6818/2528830633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb03c014ec0 con 0x7fb0280c0700 2024-09-10T22:58:07.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.281+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 --> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb044063d60 con 0x7fb0280c0700 2024-09-10T22:58:07.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.281+0000 7fb048a00640 1 -- 172.21.15.67:0/3627759981 <== osd.6 v2:172.21.15.67:6818/2528830633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb044063d60 con 0x7fb0280c0700 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] conn(0x7fb0280c0700 msgr2=0x7fb0280c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] conn(0x7fb0280c0700 0x7fb0280c2ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5440 msgr2=0x7fb0280b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0280b5440 0x7fb0280b7900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb034006010 tx=0x7fb034005e80 comp rx=0 tx=0).stop 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 msgr2=0x7fb0441a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 --2- 172.21.15.67:0/3627759981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb04410f3e0 0x7fb0441a0d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9d0 tx=0x7fb03c00ee90 comp rx=0 tx=0).stop 2024-09-10T22:58:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb043400640 1 -- 172.21.15.67:0/3627759981 reap_dead start 2024-09-10T22:58:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 shutdown_connections 2024-09-10T22:58:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 >> 172.21.15.67:0/3627759981 conn(0x7fb0440fe830 msgr2=0x7fb0440ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 shutdown_connections 2024-09-10T22:58:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:07.282+0000 7fb049e00640 1 -- 172.21.15.67:0/3627759981 wait complete. 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 -- 172.21.15.67:0/3976588308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c102da0 msgr2=0x7fec5c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 --2- 172.21.15.67:0/3976588308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c102da0 0x7fec5c103180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009900 tx=0x7fec4c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 -- 172.21.15.67:0/3976588308 shutdown_connections 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 --2- 172.21.15.67:0/3976588308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c109130 0x7fec5c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 --2- 172.21.15.67:0/3976588308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c1087b0 0x7fec5c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 --2- 172.21.15.67:0/3976588308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c102da0 0x7fec5c103180 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 -- 172.21.15.67:0/3976588308 >> 172.21.15.67:0/3976588308 conn(0x7fec5c0fe810 msgr2=0x7fec5c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 -- 172.21.15.67:0/3976588308 shutdown_connections 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.911+0000 7fec62800640 1 -- 172.21.15.67:0/3976588308 wait complete. 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.912+0000 7fec62800640 1 Processor -- start 2024-09-10T22:58:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.912+0000 7fec62800640 1 -- start start 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.912+0000 7fec62800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c102da0 0x7fec5c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec62800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c1087b0 0x7fec5c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec62800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 0x7fec5c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec62800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c112cf0 con 0x7fec5c109130 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec62800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c112b70 con 0x7fec5c1087b0 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec62800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c112e70 con 0x7fec5c102da0 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 0x7fec5c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c102da0 0x7fec5c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 0x7fec5c1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49614/0 (socket says 172.21.15.67:49614) 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5be00640 1 -- 172.21.15.67:0/937432590 learned_addr learned my addr 172.21.15.67:0/937432590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec52a00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c1087b0 0x7fec5c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 -- 172.21.15.67:0/937432590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c1087b0 msgr2=0x7fec5c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c1087b0 0x7fec5c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 -- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 msgr2=0x7fec5c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 0x7fec5c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5b400640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c1a1510 con 0x7fec5c102da0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec52a00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec5c1087b0 0x7fec5c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.913+0000 7fec5be00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec5c109130 0x7fec5c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec5b400640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c102da0 0x7fec5c19b460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec4c02a910 tx=0x7fec4c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c03f070 con 0x7fec5c102da0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec4c03baf0 con 0x7fec5c102da0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1a17a0 con 0x7fec5c102da0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c035030 con 0x7fec5c102da0 2024-09-10T22:58:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.914+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1a7ec0 con 0x7fec5c102da0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.915+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec5c0638e0 con 0x7fec5c102da0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.918+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fec4c03b660 con 0x7fec5c102da0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.919+0000 7fec58c00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec400b53c0 0x7fec400b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.919+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fec4c1071d0 con 0x7fec5c102da0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec52a00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec400b53c0 0x7fec400b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec58c00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] conn(0x7fec400c0680 0x7fec400c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec400c31f0 con 0x7fec400c0680 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fec4c107580 con 0x7fec5c102da0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec5be00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] conn(0x7fec400c0680 0x7fec400c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.920+0000 7fec52a00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec400b53c0 0x7fec400b7880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec5c108530 tx=0x7fec44005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.921+0000 7fec5be00640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] conn(0x7fec400c0680 0x7fec400c2b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.921+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== osd.9 v2:172.21.15.67:6826/3621288018 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec400c31f0 con 0x7fec400c0680 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.936+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 --> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec5c063d60 con 0x7fec400c0680 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.936+0000 7fec58c00640 1 -- 172.21.15.67:0/937432590 <== osd.9 v2:172.21.15.67:6826/3621288018 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec5c063d60 con 0x7fec400c0680 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.936+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] conn(0x7fec400c0680 msgr2=0x7fec400c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.936+0000 7fec62800640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] conn(0x7fec400c0680 0x7fec400c2b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec400b53c0 msgr2=0x7fec400b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fec400b53c0 0x7fec400b7880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec5c108530 tx=0x7fec44005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c102da0 msgr2=0x7fec5c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 --2- 172.21.15.67:0/937432590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec5c102da0 0x7fec5c19b460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec4c02a910 tx=0x7fec4c003b90 comp rx=0 tx=0).stop 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec5be00640 1 -- 172.21.15.67:0/937432590 reap_dead start 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 shutdown_connections 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 >> 172.21.15.67:0/937432590 conn(0x7fec5c0fe810 msgr2=0x7fec5c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 shutdown_connections 2024-09-10T22:58:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:06.937+0000 7fec62800640 1 -- 172.21.15.67:0/937432590 wait complete. 2024-09-10T22:58:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:07 smithi145 ceph-mon[34397]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:07.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:07 smithi067 ceph-mon[30939]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:07 smithi092 ceph-mon[34223]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:08.399 INFO:teuthology.orchestra.run.smithi067.stdout:141733920846 2024-09-10T22:58:08.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T22:58:08.537 INFO:teuthology.orchestra.run.smithi067.stdout:60129542240 2024-09-10T22:58:08.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T22:58:08.876 INFO:teuthology.orchestra.run.smithi067.stdout:115964117076 2024-09-10T22:58:08.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T22:58:09.566 INFO:teuthology.orchestra.run.smithi067.stdout:163208757321 2024-09-10T22:58:09.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T22:58:09.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:09 smithi067 ceph-mon[30939]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:09 smithi145 ceph-mon[34397]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:09 smithi092 ceph-mon[34223]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:10.295 INFO:teuthology.orchestra.run.smithi067.stdout:287762808874 2024-09-10T22:58:10.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T22:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 -- 172.21.15.67:0/3766224047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 msgr2=0x7f93d0111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 --2- 172.21.15.67:0/3766224047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d0111b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009ca0 tx=0x7f93c802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 -- 172.21.15.67:0/3766224047 shutdown_connections 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 --2- 172.21.15.67:0/3766224047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d0111b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 --2- 172.21.15.67:0/3766224047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d010a9d0 0x7f93d010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 --2- 172.21.15.67:0/3766224047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93d0104fc0 0x7f93d01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 -- 172.21.15.67:0/3766224047 >> 172.21.15.67:0/3766224047 conn(0x7f93d01009f0 msgr2=0x7f93d0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 -- 172.21.15.67:0/3766224047 shutdown_connections 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.486+0000 7f93d5200640 1 -- 172.21.15.67:0/3766224047 wait complete. 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 Processor -- start 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 -- start start 2024-09-10T22:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93d0104fc0 0x7f93d019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d010a9d0 0x7f93d019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0114e80 con 0x7f93d010b350 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0114d00 con 0x7f93d010a9d0 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.488+0000 7f93d5200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0115000 con 0x7f93d0104fc0 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d01a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53368/0 (socket says 172.21.15.67:53368) 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 -- 172.21.15.67:0/864215742 learned_addr learned my addr 172.21.15.67:0/864215742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93cde00640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93d0104fc0 0x7f93d019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93cd400640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d010a9d0 0x7f93d019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 -- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93d0104fc0 msgr2=0x7f93d019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93d0104fc0 0x7f93d019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 -- 172.21.15.67:0/864215742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d010a9d0 msgr2=0x7f93d019b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93d010a9d0 0x7f93d019b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.489+0000 7f93ce800640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d01a12e0 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93ce800640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d01a0b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93c8003ba0 tx=0x7f93c8003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004240 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93c8040070 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d01a1570 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8045da0 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.490+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a7c90 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.491+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93d010a320 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.493+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f93c8045570 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93c6a00640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f93b40b53c0 0x7f93b40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f93c81077d0 con 0x7f93d010b350 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93cde00640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f93b40b53c0 0x7f93b40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93c6a00640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] conn(0x7f93b40c0680 0x7f93b40c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93c8007e80 con 0x7f93b40c0680 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.495+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f93c8107c20 con 0x7f93d010b350 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.496+0000 7f93cde00640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f93b40b53c0 0x7f93b40b7880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005e10 tx=0x7f93c0005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.496+0000 7f93cd400640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] conn(0x7f93b40c0680 0x7f93b40c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.496+0000 7f93cd400640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] conn(0x7f93b40c0680 0x7f93b40c2b20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.497+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== osd.2 v2:172.21.15.145:6800/596498594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93c8007e80 con 0x7f93b40c0680 2024-09-10T22:58:10.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.520+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 --> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93d0106270 con 0x7f93b40c0680 2024-09-10T22:58:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.521+0000 7f93c6a00640 1 -- 172.21.15.67:0/864215742 <== osd.2 v2:172.21.15.145:6800/596498594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f93d0106270 con 0x7f93b40c0680 2024-09-10T22:58:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] conn(0x7f93b40c0680 msgr2=0x7f93b40c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] conn(0x7f93b40c0680 0x7f93b40c2b20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f93b40b53c0 msgr2=0x7f93b40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f93b40b53c0 0x7f93b40b7880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005e10 tx=0x7f93c0005da0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 msgr2=0x7f93d01a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 --2- 172.21.15.67:0/864215742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d010b350 0x7f93d01a0b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93c8003ba0 tx=0x7f93c8003bd0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93ce800640 1 -- 172.21.15.67:0/864215742 reap_dead start 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 shutdown_connections 2024-09-10T22:58:10.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 >> 172.21.15.67:0/864215742 conn(0x7f93d01009f0 msgr2=0x7f93d0103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 shutdown_connections 2024-09-10T22:58:10.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:10.522+0000 7f93d5200640 1 -- 172.21.15.67:0/864215742 wait complete. 2024-09-10T22:58:11.516 INFO:teuthology.orchestra.run.smithi067.stdout:81604378717 2024-09-10T22:58:11.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T22:58:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:11 smithi067 ceph-mon[30939]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:11.879 INFO:teuthology.orchestra.run.smithi067.stdout:188978561091 2024-09-10T22:58:11.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T22:58:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:11 smithi145 ceph-mon[34397]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:11 smithi092 ceph-mon[34223]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.125+0000 7f4742800640 1 -- 172.21.15.67:0/872328933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c075e00 msgr2=0x7f473c10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.125+0000 7f4742800640 1 --2- 172.21.15.67:0/872328933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c075e00 0x7f473c10ee70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4724009900 tx=0x7f472402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 -- 172.21.15.67:0/872328933 shutdown_connections 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 --2- 172.21.15.67:0/872328933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c10f3b0 0x7f473c1117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 --2- 172.21.15.67:0/872328933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c075e00 0x7f473c10ee70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 --2- 172.21.15.67:0/872328933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c0758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 -- 172.21.15.67:0/872328933 >> 172.21.15.67:0/872328933 conn(0x7f473c0fe830 msgr2=0x7f473c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 -- 172.21.15.67:0/872328933 shutdown_connections 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.126+0000 7f4742800640 1 -- 172.21.15.67:0/872328933 wait complete. 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.127+0000 7f4742800640 1 Processor -- start 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.127+0000 7f4742800640 1 -- start start 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.127+0000 7f4742800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.127+0000 7f4742800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c075e00 0x7f473c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f4742800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c10f3b0 0x7f473c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f4742800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f473c114e20 con 0x7f473c10f3b0 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f4742800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f473c114ca0 con 0x7f473c075e00 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f4742800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f473c114fa0 con 0x7f473c0754e0 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f473b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f473b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43016/0 (socket says 172.21.15.67:43016) 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f473b400640 1 -- 172.21.15.67:0/1767682219 learned_addr learned my addr 172.21.15.67:0/1767682219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f473be00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c10f3b0 0x7f473c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.128+0000 7f4733e00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c075e00 0x7f473c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 -- 172.21.15.67:0/1767682219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c075e00 msgr2=0x7f473c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c075e00 0x7f473c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 -- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c10f3b0 msgr2=0x7f473c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c10f3b0 0x7f473c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1a14e0 con 0x7f473c0754e0 2024-09-10T22:58:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f473b400640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c19b430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f472c00d770 tx=0x7f472c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c017070 con 0x7f473c0754e0 2024-09-10T22:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.129+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f472c0054b0 con 0x7f473c0754e0 2024-09-10T22:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.130+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c1a17d0 con 0x7f473c0754e0 2024-09-10T22:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.130+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c004c60 con 0x7f473c0754e0 2024-09-10T22:58:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.130+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a7e90 con 0x7f473c0754e0 2024-09-10T22:58:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.131+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f473c0638a0 con 0x7f473c0754e0 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.133+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f472c004e00 con 0x7f473c0754e0 2024-09-10T22:58:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4738c00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47200b5440 0x7f47200b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f472c0e5b70 con 0x7f473c0754e0 2024-09-10T22:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4733e00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47200b5440 0x7f47200b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4738c00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] conn(0x7f47200c0700 0x7f47200c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47200c3270 con 0x7f47200c0700 2024-09-10T22:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.136+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f472c0ea2a0 con 0x7f473c0754e0 2024-09-10T22:58:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.137+0000 7f4733e00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47200b5440 0x7f47200b7900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47240057c0 tx=0x7f47240058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.137+0000 7f473be00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] conn(0x7f47200c0700 0x7f47200c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.138+0000 7f473be00640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] conn(0x7f47200c0700 0x7f47200c2ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.139+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== osd.11 v2:172.21.15.145:6824/257658495 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47200c3270 con 0x7f47200c0700 2024-09-10T22:58:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.156+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 --> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f473c063d50 con 0x7f47200c0700 2024-09-10T22:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.157+0000 7f4738c00640 1 -- 172.21.15.67:0/1767682219 <== osd.11 v2:172.21.15.145:6824/257658495 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f473c063d50 con 0x7f47200c0700 2024-09-10T22:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.157+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] conn(0x7f47200c0700 msgr2=0x7f47200c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.157+0000 7f4742800640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] conn(0x7f47200c0700 0x7f47200c2ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47200b5440 msgr2=0x7f47200b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47200b5440 0x7f47200b7900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47240057c0 tx=0x7f47240058d0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 msgr2=0x7f473c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 --2- 172.21.15.67:0/1767682219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f473c0754e0 0x7f473c19b430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f472c00d770 tx=0x7f472c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f473be00640 1 -- 172.21.15.67:0/1767682219 reap_dead start 2024-09-10T22:58:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 shutdown_connections 2024-09-10T22:58:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 >> 172.21.15.67:0/1767682219 conn(0x7f473c0fe830 msgr2=0x7f473c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 shutdown_connections 2024-09-10T22:58:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:12.158+0000 7f4742800640 1 -- 172.21.15.67:0/1767682219 wait complete. 2024-09-10T22:58:12.922 INFO:teuthology.orchestra.run.smithi067.stdout:266287972401 2024-09-10T22:58:12.922 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T22:58:13.123 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:13.124 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:13.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 -- 172.21.15.67:0/1560908036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e200776d0 msgr2=0x7f9e20077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 --2- 172.21.15.67:0/1560908036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e200776d0 0x7f9e20077ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009900 tx=0x7f9e1002a400 comp rx=0 tx=0).stop 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 -- 172.21.15.67:0/1560908036 shutdown_connections 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 --2- 172.21.15.67:0/1560908036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e201139b0 0x7f9e20115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 --2- 172.21.15.67:0/1560908036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e20077ff0 0x7f9e20113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 --2- 172.21.15.67:0/1560908036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e200776d0 0x7f9e20077ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 -- 172.21.15.67:0/1560908036 >> 172.21.15.67:0/1560908036 conn(0x7f9e201009f0 msgr2=0x7f9e20102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.384+0000 7f9e24a00640 1 -- 172.21.15.67:0/1560908036 shutdown_connections 2024-09-10T22:58:13.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.386+0000 7f9e24a00640 1 -- 172.21.15.67:0/1560908036 wait complete. 2024-09-10T22:58:13.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.387+0000 7f9e24a00640 1 Processor -- start 2024-09-10T22:58:13.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 -- start start 2024-09-10T22:58:13.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e200776d0 0x7f9e2019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e20077ff0 0x7f9e2019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 0x7f9e201a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e201192a0 con 0x7f9e20077ff0 2024-09-10T22:58:13.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e20119120 con 0x7f9e200776d0 2024-09-10T22:58:13.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.388+0000 7f9e24a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e20119420 con 0x7f9e201139b0 2024-09-10T22:58:13.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.389+0000 7f9e1e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 0x7f9e201a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.389+0000 7f9e1e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 0x7f9e201a5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43032/0 (socket says 172.21.15.67:43032) 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.389+0000 7f9e1e800640 1 -- 172.21.15.67:0/4289751836 learned_addr learned my addr 172.21.15.67:0/4289751836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.390+0000 7f9e1e800640 1 -- 172.21.15.67:0/4289751836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e200776d0 msgr2=0x7f9e2019f770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.390+0000 7f9e1e800640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e200776d0 0x7f9e2019f770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.390+0000 7f9e1e800640 1 -- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e20077ff0 msgr2=0x7f9e2019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.390+0000 7f9e1e800640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e20077ff0 0x7f9e2019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.390+0000 7f9e1e800640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e201a5820 con 0x7f9e201139b0 2024-09-10T22:58:13.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.391+0000 7f9e1e800640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 0x7f9e201a5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb40 tx=0x7f9e1800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.391+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18004b20 con 0x7f9e201139b0 2024-09-10T22:58:13.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.391+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e201a5b10 con 0x7f9e201139b0 2024-09-10T22:58:13.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.391+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e18004cc0 con 0x7f9e201139b0 2024-09-10T22:58:13.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.391+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e201ac1d0 con 0x7f9e201139b0 2024-09-10T22:58:13.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.392+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18010690 con 0x7f9e201139b0 2024-09-10T22:58:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.394+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e00001060 con 0x7f9e201139b0 2024-09-10T22:58:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.398+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f9e18005020 con 0x7f9e201139b0 2024-09-10T22:58:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.401+0000 7f9e16a00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e040b5440 0x7f9e040b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.401+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f9e180e6910 con 0x7f9e201139b0 2024-09-10T22:58:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.402+0000 7f9e1de00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e040b5440 0x7f9e040b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.402+0000 7f9e16a00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] conn(0x7f9e040c0700 0x7f9e040c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.402+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e180047f0 con 0x7f9e040c0700 2024-09-10T22:58:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.402+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9e180eb2c0 con 0x7f9e201139b0 2024-09-10T22:58:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.403+0000 7f9e1d400640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] conn(0x7f9e040c0700 0x7f9e040c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.403+0000 7f9e1de00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e040b5440 0x7f9e040b7900 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e100096c0 tx=0x7f9e10005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.404+0000 7f9e1d400640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] conn(0x7f9e040c0700 0x7f9e040c2ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.405+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== osd.8 v2:172.21.15.145:6816/1197192386 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e180047f0 con 0x7f9e040c0700 2024-09-10T22:58:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.436+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 --> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e00002e00 con 0x7f9e040c0700 2024-09-10T22:58:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.437+0000 7f9e16a00640 1 -- 172.21.15.67:0/4289751836 <== osd.8 v2:172.21.15.145:6816/1197192386 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9e00002e00 con 0x7f9e040c0700 2024-09-10T22:58:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.437+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] conn(0x7f9e040c0700 msgr2=0x7f9e040c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.437+0000 7f9e24a00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] conn(0x7f9e040c0700 0x7f9e040c2ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.438+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e040b5440 msgr2=0x7f9e040b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.438+0000 7f9e24a00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e040b5440 0x7f9e040b7900 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e100096c0 tx=0x7f9e10005fd0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.438+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 msgr2=0x7f9e201a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.438+0000 7f9e24a00640 1 --2- 172.21.15.67:0/4289751836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e201139b0 0x7f9e201a5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb40 tx=0x7f9e1800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.439+0000 7f9e1e800640 1 -- 172.21.15.67:0/4289751836 reap_dead start 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.439+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 shutdown_connections 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.439+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 >> 172.21.15.67:0/4289751836 conn(0x7f9e201009f0 msgr2=0x7f9e20102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.440+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 shutdown_connections 2024-09-10T22:58:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:13.440+0000 7f9e24a00640 1 -- 172.21.15.67:0/4289751836 wait complete. 2024-09-10T22:58:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:13 smithi067 ceph-mon[30939]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:13 smithi145 ceph-mon[34397]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:13 smithi092 ceph-mon[34223]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:14.255 INFO:teuthology.orchestra.run.smithi067.stdout:240518168633 2024-09-10T22:58:14.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T22:58:15.546 INFO:teuthology.orchestra.run.smithi067.stdout:313532612646 2024-09-10T22:58:15.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T22:58:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:15 smithi067 ceph-mon[30939]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:15 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3593076197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:58:15.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 -- 172.21.15.67:0/4257051367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2424104e10 msgr2=0x7f24241051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 --2- 172.21.15.67:0/4257051367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2424104e10 0x7f24241051f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2418009900 tx=0x7f241802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 -- 172.21.15.67:0/4257051367 shutdown_connections 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 --2- 172.21.15.67:0/4257051367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242410b1a0 0x7f24241119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 --2- 172.21.15.67:0/4257051367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410a820 0x7f242410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 --2- 172.21.15.67:0/4257051367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2424104e10 0x7f24241051f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 -- 172.21.15.67:0/4257051367 >> 172.21.15.67:0/4257051367 conn(0x7f2424100840 msgr2=0x7f2424102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 -- 172.21.15.67:0/4257051367 shutdown_connections 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.005+0000 7f242b200640 1 -- 172.21.15.67:0/4257051367 wait complete. 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.006+0000 7f242b200640 1 Processor -- start 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.006+0000 7f242b200640 1 -- start start 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2424104e10 0x7f242419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 0x7f24241a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2424114d60 con 0x7f242410a820 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2424114be0 con 0x7f2424104e10 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2424114ee0 con 0x7f242410b1a0 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53436/0 (socket says 172.21.15.67:53436) 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 -- 172.21.15.67:0/3593076197 learned_addr learned my addr 172.21.15.67:0/3593076197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429e00640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2424104e10 0x7f242419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242a800640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 0x7f24241a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 -- 172.21.15.67:0/3593076197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 msgr2=0x7f24241a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 0x7f24241a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 -- 172.21.15.67:0/3593076197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2424104e10 msgr2=0x7f242419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2424104e10 0x7f242419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f2429400640 1 -- 172.21.15.67:0/3593076197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24241a12a0 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.007+0000 7f242a800640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 0x7f24241a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f2429400640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f241400d950 tx=0x7f241400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414017070 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2414009c30 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400f620 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241a1590 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.008+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241a7c50 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.009+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24241051f0 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.013+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f2414010460 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.014+0000 7f241ea00640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 0x7f240c0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.014+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f24140e5f30 con 0x7f242410a820 2024-09-10T22:58:15.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.015+0000 7f2429e00640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 0x7f240c0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.865 INFO:teuthology.orchestra.run.smithi067.stdout:38654705768 2024-09-10T22:58:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.015+0000 7f2429e00640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 0x7f240c0b7920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2418006010 tx=0x7f2418005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.016+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f24140ac810 con 0x7f242410a820 2024-09-10T22:58:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.228+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f242419c320 con 0x7f242410a820 2024-09-10T22:58:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.229+0000 7f241ea00640 1 -- 172.21.15.67:0/3593076197 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f242419c320 con 0x7f242410a820 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 msgr2=0x7f240c0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 0x7f240c0b7920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2418006010 tx=0x7f2418005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 msgr2=0x7f242419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f241400d950 tx=0x7f241400de10 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 shutdown_connections 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f240c0b5460 0x7f240c0b7920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f242410b1a0 0x7f24241a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242410a820 0x7f242419b730 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 --2- 172.21.15.67:0/3593076197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2424104e10 0x7f242419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 >> 172.21.15.67:0/3593076197 conn(0x7f2424100840 msgr2=0x7f2424102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 shutdown_connections 2024-09-10T22:58:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:15.231+0000 7f242b200640 1 -- 172.21.15.67:0/3593076197 wait complete. 2024-09-10T22:58:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:15 smithi145 ceph-mon[34397]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:15 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3593076197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:58:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:15 smithi092 ceph-mon[34223]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:15 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3593076197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:58:16.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 -- 172.21.15.67:0/2573293753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8108fa0 msgr2=0x7fccf810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:16.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 --2- 172.21.15.67:0/2573293753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8108fa0 0x7fccf810f7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009ca0 tx=0x7fccf002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 -- 172.21.15.67:0/2573293753 shutdown_connections 2024-09-10T22:58:16.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 --2- 172.21.15.67:0/2573293753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8108fa0 0x7fccf810f7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 --2- 172.21.15.67:0/2573293753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 --2- 172.21.15.67:0/2573293753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8102c10 0x7fccf8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 -- 172.21.15.67:0/2573293753 >> 172.21.15.67:0/2573293753 conn(0x7fccf80fe660 msgr2=0x7fccf8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 -- 172.21.15.67:0/2573293753 shutdown_connections 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.386+0000 7fccfd600640 1 -- 172.21.15.67:0/2573293753 wait complete. 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 Processor -- start 2024-09-10T22:58:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 -- start start 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8108fa0 0x7fccf819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8112b20 con 0x7fccf8102c10 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81129a0 con 0x7fccf8108fa0 2024-09-10T22:58:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.388+0000 7fccfd600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8112ca0 con 0x7fccf8108620 2024-09-10T22:58:16.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53454/0 (socket says 172.21.15.67:53454) 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 -- 172.21.15.67:0/2252267965 learned_addr learned my addr 172.21.15.67:0/2252267965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccfcc00640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8108fa0 0x7fccf819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7400640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 -- 172.21.15.67:0/2252267965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 msgr2=0x7fccf8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 -- 172.21.15.67:0/2252267965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8108fa0 msgr2=0x7fccf819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8108fa0 0x7fccf819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 -- 172.21.15.67:0/2252267965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf819efe0 con 0x7fccf8102c10 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7400640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.389+0000 7fccf7e00640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.390+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8017070 con 0x7fccf8102c10 2024-09-10T22:58:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.390+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce8009c30 con 0x7fccf8102c10 2024-09-10T22:58:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.390+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf819f2d0 con 0x7fccf8102c10 2024-09-10T22:58:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.390+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce800f620 con 0x7fccf8102c10 2024-09-10T22:58:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.390+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a5b60 con 0x7fccf8102c10 2024-09-10T22:58:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.391+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8102ff0 con 0x7fccf8102c10 2024-09-10T22:58:16.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.395+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fcce8010460 con 0x7fccf8102c10 2024-09-10T22:58:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.396+0000 7fccf4c00640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 0x7fccdc0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.397+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fcce80e5da0 con 0x7fccf8102c10 2024-09-10T22:58:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.397+0000 7fccf7400640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 0x7fccdc0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.397+0000 7fccf7400640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 0x7fccdc0b79a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccf8101080 tx=0x7fcce000b5f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.398+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcce80ac760 con 0x7fccf8102c10 2024-09-10T22:58:16.491 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.495 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.495 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.502 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.529 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.572 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.684+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fccf81a5d30 con 0x7fccf8102c10 2024-09-10T22:58:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.684+0000 7fccf4c00640 1 -- 172.21.15.67:0/2252267965 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcce8009de0 con 0x7fccf8102c10 2024-09-10T22:58:16.686 INFO:teuthology.orchestra.run.smithi067.stdout:210453397566 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 msgr2=0x7fccdc0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 0x7fccdc0b79a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccf8101080 tx=0x7fcce000b5f0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 msgr2=0x7fccf8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 shutdown_connections 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fccdc0b54e0 0x7fccdc0b79a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccf8108fa0 0x7fccf819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8108620 0x7fccf8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 --2- 172.21.15.67:0/2252267965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf8102c10 0x7fccf8198f30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.687+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 >> 172.21.15.67:0/2252267965 conn(0x7fccf80fe660 msgr2=0x7fccf80fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:16.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.688+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 shutdown_connections 2024-09-10T22:58:16.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:16.688+0000 7fccfd600640 1 -- 172.21.15.67:0/2252267965 wait complete. 2024-09-10T22:58:17.122 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705766 got 38654705768 for osd.0 2024-09-10T22:58:17.123 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:17.335 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:17.338 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:17.732 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:17 smithi067 ceph-mon[30939]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:17.732 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2252267965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:58:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:17 smithi145 ceph-mon[34397]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2252267965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:58:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:17 smithi092 ceph-mon[34223]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2252267965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:58:18.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 -- 172.21.15.67:0/3236140496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c075f40 msgr2=0x7fc57c10edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 --2- 172.21.15.67:0/3236140496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c075f40 0x7fc57c10edf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc564009900 tx=0x7fc56402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 -- 172.21.15.67:0/3236140496 shutdown_connections 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 --2- 172.21.15.67:0/3236140496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 --2- 172.21.15.67:0/3236140496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c075f40 0x7fc57c10edf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 --2- 172.21.15.67:0/3236140496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075540 0x7fc57c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.663+0000 7fc583c00640 1 -- 172.21.15.67:0/3236140496 >> 172.21.15.67:0/3236140496 conn(0x7fc57c0fe810 msgr2=0x7fc57c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.664+0000 7fc583c00640 1 -- 172.21.15.67:0/3236140496 shutdown_connections 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.664+0000 7fc583c00640 1 -- 172.21.15.67:0/3236140496 wait complete. 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.664+0000 7fc583c00640 1 Processor -- start 2024-09-10T22:58:18.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 -- start start 2024-09-10T22:58:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 0x7fc57c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 0x7fc57c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c114e10 con 0x7fc57c10f3c0 2024-09-10T22:58:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c114c90 con 0x7fc57c075f40 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc583c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c114f90 con 0x7fc57c075540 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc581c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc581200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 0x7fc57c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc573e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 0x7fc57c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc581200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 0x7fc57c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43106/0 (socket says 172.21.15.67:43106) 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc581200640 1 -- 172.21.15.67:0/715681893 learned_addr learned my addr 172.21.15.67:0/715681893 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.665+0000 7fc573e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 0x7fc57c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:37656/0 (socket says 172.21.15.67:37656) 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 -- 172.21.15.67:0/715681893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 msgr2=0x7fc57c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 0x7fc57c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 -- 172.21.15.67:0/715681893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 msgr2=0x7fc57c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 0x7fc57c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 -- 172.21.15.67:0/715681893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57c1a1430 con 0x7fc57c10f3c0 2024-09-10T22:58:18.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc581c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1a0c60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc57400ca80 tx=0x7fc57400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.666+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57400ae20 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.667+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc57400f8e0 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.667+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57c1a1720 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.667+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740189a0 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.667+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57c1a7de0 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.668+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57c075920 con 0x7fc57c10f3c0 2024-09-10T22:58:18.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.670+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc574020030 con 0x7fc57c10f3c0 2024-09-10T22:58:18.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.671+0000 7fc57a000640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 0x7fc5600b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.671+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fc5740e6170 con 0x7fc57c10f3c0 2024-09-10T22:58:18.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.672+0000 7fc581200640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 0x7fc5600b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.672+0000 7fc581200640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 0x7fc5600b7880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00ac90 tx=0x7fc56c00a380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.675+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc574014030 con 0x7fc57c10f3c0 2024-09-10T22:58:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.873+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc57c1a8120 con 0x7fc57c10f3c0 2024-09-10T22:58:18.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.873+0000 7fc57a000640 1 -- 172.21.15.67:0/715681893 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc5740b5070 con 0x7fc57c10f3c0 2024-09-10T22:58:18.875 INFO:teuthology.orchestra.run.smithi067.stdout:163208757323 2024-09-10T22:58:18.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.876+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 msgr2=0x7fc5600b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.876+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 0x7fc5600b7880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00ac90 tx=0x7fc56c00a380 comp rx=0 tx=0).stop 2024-09-10T22:58:18.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.876+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 msgr2=0x7fc57c1a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.876+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1a0c60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc57400ca80 tx=0x7fc57400cf40 comp rx=0 tx=0).stop 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 shutdown_connections 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc5600b53c0 0x7fc5600b7880 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc57c10f3c0 0x7fc57c1a0c60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c075f40 0x7fc57c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 --2- 172.21.15.67:0/715681893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc57c075540 0x7fc57c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 >> 172.21.15.67:0/715681893 conn(0x7fc57c0fe810 msgr2=0x7fc57c111580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 shutdown_connections 2024-09-10T22:58:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.877+0000 7fc583c00640 1 -- 172.21.15.67:0/715681893 wait complete. 2024-09-10T22:58:18.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 -- 172.21.15.67:0/3757458581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d00776d0 msgr2=0x7f33d0077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 --2- 172.21.15.67:0/3757458581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d00776d0 0x7f33d0077ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009900 tx=0x7f33c002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 -- 172.21.15.67:0/3757458581 shutdown_connections 2024-09-10T22:58:18.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 --2- 172.21.15.67:0/3757458581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d0113980 0x7f33d0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 --2- 172.21.15.67:0/3757458581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d0113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 --2- 172.21.15.67:0/3757458581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d00776d0 0x7f33d0077ab0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 -- 172.21.15.67:0/3757458581 >> 172.21.15.67:0/3757458581 conn(0x7f33d0100a00 msgr2=0x7f33d0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 -- 172.21.15.67:0/3757458581 shutdown_connections 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.881+0000 7f33d6200640 1 -- 172.21.15.67:0/3757458581 wait complete. 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.882+0000 7f33d6200640 1 Processor -- start 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.882+0000 7f33d6200640 1 -- start start 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d00776d0 0x7f33d019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 0x7f33d01a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0119240 con 0x7f33d0113980 2024-09-10T22:58:18.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01190c0 con 0x7f33d00776d0 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33d6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01193c0 con 0x7f33d0077ff0 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 0x7f33d01a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:50992/0 (socket says 172.21.15.67:50992) 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33c7e00640 1 -- 172.21.15.67:0/626459133 learned_addr learned my addr 172.21.15.67:0/626459133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.883+0000 7f33cf400640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d00776d0 0x7f33d019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 -- 172.21.15.67:0/626459133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d00776d0 msgr2=0x7f33d019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d00776d0 0x7f33d019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 -- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 msgr2=0x7f33d01a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 0x7f33d01a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 -- 172.21.15.67:0/626459133 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d01a58c0 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33cfe00640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 0x7f33d01a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33c7e00640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e9f0 tx=0x7f33b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b800cd80 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d01a5bb0 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33b80054b0 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.884+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8010430 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.885+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01ac270 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.886+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d0077ab0 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.890+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f33b8005020 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.892+0000 7f33ccc00640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 0x7f33b40b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.892+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f33b80e64f0 con 0x7f33d0077ff0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.893+0000 7f33cf400640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 0x7f33b40b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.893+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f33b80e6a10 con 0x7f33d0077ff0 2024-09-10T22:58:18.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:18.893+0000 7f33cf400640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 0x7f33b40b7900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f33c00057c0 tx=0x7f33c00058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.100+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f33d01a0940 con 0x7f33d0077ff0 2024-09-10T22:58:19.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.101+0000 7f33ccc00640 1 -- 172.21.15.67:0/626459133 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f33b80ace30 con 0x7f33d0077ff0 2024-09-10T22:58:19.103 INFO:teuthology.orchestra.run.smithi067.stdout:141733920849 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.103+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 msgr2=0x7f33b40b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.103+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 0x7f33b40b7900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f33c00057c0 tx=0x7f33c00058d0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.103+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 msgr2=0x7f33d019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.103+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e9f0 tx=0x7f33b800edf0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 shutdown_connections 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33b40b5440 0x7f33b40b7900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d0113980 0x7f33d01a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33d0077ff0 0x7f33d019fd50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 --2- 172.21.15.67:0/626459133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33d00776d0 0x7f33d019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 >> 172.21.15.67:0/626459133 conn(0x7f33d0100a00 msgr2=0x7f33d0102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 shutdown_connections 2024-09-10T22:58:19.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:19.104+0000 7f33d6200640 1 -- 172.21.15.67:0/626459133 wait complete. 2024-09-10T22:58:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:19 smithi067 ceph-mon[30939]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/715681893' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:58:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/626459133' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:58:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:19 smithi145 ceph-mon[34397]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/715681893' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:58:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/626459133' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:19 smithi092 ceph-mon[34223]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/715681893' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:58:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/626459133' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:58:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 -- 172.21.15.67:0/2758810538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028108f70 msgr2=0x7f602810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 --2- 172.21.15.67:0/2758810538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028108f70 0x7f602810f770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6020009ca0 tx=0x7f602002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 -- 172.21.15.67:0/2758810538 shutdown_connections 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 --2- 172.21.15.67:0/2758810538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028108f70 0x7f602810f770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 --2- 172.21.15.67:0/2758810538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60281085f0 0x7f6028108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 --2- 172.21.15.67:0/2758810538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6028102be0 0x7f6028102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 -- 172.21.15.67:0/2758810538 >> 172.21.15.67:0/2758810538 conn(0x7f60280fe610 msgr2=0x7f6028100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 -- 172.21.15.67:0/2758810538 shutdown_connections 2024-09-10T22:58:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.528+0000 7f602f000640 1 -- 172.21.15.67:0/2758810538 wait complete. 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.529+0000 7f602f000640 1 Processor -- start 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.529+0000 7f602f000640 1 -- start start 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60281085f0 0x7f602819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028108f70 0x7f60281a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6028112b30 con 0x7f6028102be0 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f60281129b0 con 0x7f6028108f70 2024-09-10T22:58:20.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602f000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6028112cb0 con 0x7f60281085f0 2024-09-10T22:58:20.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:20.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51654/0 (socket says 172.21.15.67:51654) 2024-09-10T22:58:20.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602dc00640 1 -- 172.21.15.67:0/3883307189 learned_addr learned my addr 172.21.15.67:0/3883307189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:20.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602d200640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60281085f0 0x7f602819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:20.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.530+0000 7f602e600640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028108f70 0x7f60281a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 -- 172.21.15.67:0/3883307189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60281085f0 msgr2=0x7f602819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60281085f0 0x7f602819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 -- 172.21.15.67:0/3883307189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028108f70 msgr2=0x7f60281a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028108f70 0x7f60281a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 -- 172.21.15.67:0/3883307189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a11b0 con 0x7f6028102be0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602dc00640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c00cd80 con 0x7f6028102be0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f601c0054b0 con 0x7f6028102be0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c010430 con 0x7f6028102be0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.531+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a14a0 con 0x7f6028102be0 2024-09-10T22:58:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.532+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a7b60 con 0x7f6028102be0 2024-09-10T22:58:20.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.533+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6028102fc0 con 0x7f6028102be0 2024-09-10T22:58:20.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.535+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f601c005020 con 0x7f6028102be0 2024-09-10T22:58:20.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.536+0000 7f6026000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 0x7f60100b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:20.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.537+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f601c0e6690 con 0x7f6028102be0 2024-09-10T22:58:20.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.537+0000 7f602d200640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 0x7f60100b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:20.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.537+0000 7f602d200640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 0x7f60100b79a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6028108150 tx=0x7f601400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:20.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.540+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f601c0acf70 con 0x7f6028102be0 2024-09-10T22:58:20.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.737+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f602810ebd0 con 0x7f6028102be0 2024-09-10T22:58:20.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.738+0000 7f6026000640 1 -- 172.21.15.67:0/3883307189 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f601c0ac870 con 0x7f6028102be0 2024-09-10T22:58:20.740 INFO:teuthology.orchestra.run.smithi067.stdout:115964117079 2024-09-10T22:58:20.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.740+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 msgr2=0x7f60100b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:20.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.740+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 0x7f60100b79a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6028108150 tx=0x7f601400b390 comp rx=0 tx=0).stop 2024-09-10T22:58:20.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.740+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 msgr2=0x7f602819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.740+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00edf0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 shutdown_connections 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f60100b54e0 0x7f60100b79a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028108f70 0x7f60281a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60281085f0 0x7f602819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 --2- 172.21.15.67:0/3883307189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6028102be0 0x7f602819b100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 >> 172.21.15.67:0/3883307189 conn(0x7f60280fe610 msgr2=0x7f6028104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 shutdown_connections 2024-09-10T22:58:20.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.741+0000 7f602f000640 1 -- 172.21.15.67:0/3883307189 wait complete. 2024-09-10T22:58:21.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 -- 172.21.15.67:0/1582179441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010a9d0 msgr2=0x7f62c010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 --2- 172.21.15.67:0/1582179441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010a9d0 0x7f62c010ae10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009900 tx=0x7f62a802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 -- 172.21.15.67:0/1582179441 shutdown_connections 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 --2- 172.21.15.67:0/1582179441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c010b350 0x7f62c0111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 --2- 172.21.15.67:0/1582179441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010a9d0 0x7f62c010ae10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 --2- 172.21.15.67:0/1582179441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c0104fc0 0x7f62c01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 -- 172.21.15.67:0/1582179441 >> 172.21.15.67:0/1582179441 conn(0x7f62c01009f0 msgr2=0x7f62c0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 -- 172.21.15.67:0/1582179441 shutdown_connections 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.882+0000 7f62c7a00640 1 -- 172.21.15.67:0/1582179441 wait complete. 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 Processor -- start 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 -- start start 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c010a9d0 0x7f62c019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 0x7f62c01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0114e80 con 0x7f62c010b350 2024-09-10T22:58:21.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0114d00 con 0x7f62c010a9d0 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.883+0000 7f62c7a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0115000 con 0x7f62c0104fc0 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 0x7f62c01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51014/0 (socket says 172.21.15.67:51014) 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 -- 172.21.15.67:0/3316203976 learned_addr learned my addr 172.21.15.67:0/3316203976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62bfe00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c010a9d0 0x7f62c019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 -- 172.21.15.67:0/3316203976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c010a9d0 msgr2=0x7f62c019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c010a9d0 0x7f62c019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 -- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 msgr2=0x7f62c01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stdout:313532612648 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 0x7f62c01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5000640 1 -- 172.21.15.67:0/3316203976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c01a14e0 con 0x7f62c0104fc0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.884+0000 7f62c5a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 0x7f62c01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62c5000640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62b000e9f0 tx=0x7f62b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000cd80 con 0x7f62c0104fc0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01a17d0 con 0x7f62c0104fc0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62b00108c0 con 0x7f62c0104fc0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00187c0 con 0x7f62c0104fc0 2024-09-10T22:58:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.885+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01a8060 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.886+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c0106270 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.888+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f62b0010430 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.890+0000 7f62bd600640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 0x7f62a40b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.890+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f62b00e6980 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.890+0000 7f62bfe00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 0x7f62a40b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.891+0000 7f62bfe00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 0x7f62a40b7900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62a8006010 tx=0x7f62a8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:20.891+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f62b00acc10 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.090+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f62c010d160 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.090+0000 7f62bd600640 1 -- 172.21.15.67:0/3316203976 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f62b00b1640 con 0x7f62c0104fc0 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.092+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 msgr2=0x7f62a40b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.092+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 0x7f62a40b7900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62a8006010 tx=0x7f62a8005e80 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.092+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 msgr2=0x7f62c019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.092+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62b000e9f0 tx=0x7f62b000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 shutdown_connections 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f62a40b5440 0x7f62a40b7900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c010b350 0x7f62c01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c010a9d0 0x7f62c019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 --2- 172.21.15.67:0/3316203976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0104fc0 0x7f62c019b430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 >> 172.21.15.67:0/3316203976 conn(0x7f62c01009f0 msgr2=0x7f62c0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 shutdown_connections 2024-09-10T22:58:21.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.093+0000 7f62c7a00640 1 -- 172.21.15.67:0/3316203976 wait complete. 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 -- 172.21.15.67:0/2001345270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c104fa0 msgr2=0x7fd31c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 --2- 172.21.15.67:0/2001345270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c104fa0 0x7fd31c105380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009900 tx=0x7fd30c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 -- 172.21.15.67:0/2001345270 shutdown_connections 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 --2- 172.21.15.67:0/2001345270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c10b330 0x7fd31c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 --2- 172.21.15.67:0/2001345270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10a9b0 0x7fd31c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 --2- 172.21.15.67:0/2001345270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c104fa0 0x7fd31c105380 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 -- 172.21.15.67:0/2001345270 >> 172.21.15.67:0/2001345270 conn(0x7fd31c100a50 msgr2=0x7fd31c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 -- 172.21.15.67:0/2001345270 shutdown_connections 2024-09-10T22:58:21.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.569+0000 7fd322600640 1 -- 172.21.15.67:0/2001345270 wait complete. 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.570+0000 7fd322600640 1 Processor -- start 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 -- start start 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 0x7fd31c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10b330 0x7fd31c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c114e10 con 0x7fd31c10a9b0 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c114c90 con 0x7fd31c10b330 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd322600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c114f90 con 0x7fd31c104fa0 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd31b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd31aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 0x7fd31c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd31b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51032/0 (socket says 172.21.15.67:51032) 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd31b400640 1 -- 172.21.15.67:0/633593148 learned_addr learned my addr 172.21.15.67:0/633593148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.571+0000 7fd31be00640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10b330 0x7fd31c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 -- 172.21.15.67:0/633593148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10b330 msgr2=0x7fd31c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10b330 0x7fd31c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 -- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 msgr2=0x7fd31c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 0x7fd31c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 -- 172.21.15.67:0/633593148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1a1510 con 0x7fd31c104fa0 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31aa00640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 0x7fd31c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.572+0000 7fd31b400640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009230 tx=0x7fd30c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.573+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c03f030 con 0x7fd31c104fa0 2024-09-10T22:58:21.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.573+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd30c003a80 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.573+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c035070 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.573+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c1a17a0 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.573+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1a7ec0 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.574+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c106250 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.577+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fd30c003c30 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.578+0000 7fd313e00640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 0x7fd3000b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.578+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fd30c106e70 con 0x7fd31c104fa0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.579+0000 7fd31aa00640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 0x7fd3000b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.579+0000 7fd31aa00640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 0x7fd3000b7900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd31c10a670 tx=0x7fd308005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.580+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd30c0cd100 con 0x7fd31c104fa0 2024-09-10T22:58:21.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:21 smithi067 ceph-mon[30939]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:21.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:21 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3883307189' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:58:21.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:21 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3316203976' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:58:21.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:21.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.830+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd31c105380 con 0x7fd31c104fa0 2024-09-10T22:58:21.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.830+0000 7fd313e00640 1 -- 172.21.15.67:0/633593148 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd30c0d1b30 con 0x7fd31c104fa0 2024-09-10T22:58:21.833 INFO:teuthology.orchestra.run.smithi067.stdout:81604378719 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 msgr2=0x7fd3000b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 0x7fd3000b7900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd31c10a670 tx=0x7fd308005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 msgr2=0x7fd31c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009230 tx=0x7fd30c004270 comp rx=0 tx=0).stop 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 shutdown_connections 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3000b5440 0x7fd3000b7900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c10b330 0x7fd31c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c10a9b0 0x7fd31c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 --2- 172.21.15.67:0/633593148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd31c104fa0 0x7fd31c19b460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 >> 172.21.15.67:0/633593148 conn(0x7fd31c100a50 msgr2=0x7fd31c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 shutdown_connections 2024-09-10T22:58:21.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.833+0000 7fd322600640 1 -- 172.21.15.67:0/633593148 wait complete. 2024-09-10T22:58:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:21 smithi145 ceph-mon[34397]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:21 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3883307189' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:58:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:21 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3316203976' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:58:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:21 smithi092 ceph-mon[34223]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:21 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3883307189' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:21 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3316203976' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:58:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 -- 172.21.15.67:0/3734158741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 msgr2=0x7f73f010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 --2- 172.21.15.67:0/3734158741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f010f3d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ca0 tx=0x7f73e802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 -- 172.21.15.67:0/3734158741 shutdown_connections 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 --2- 172.21.15.67:0/3734158741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f010f3d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 --2- 172.21.15.67:0/3734158741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0104430 0x7f73f010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 --2- 172.21.15.67:0/3734158741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0103b10 0x7f73f0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.735+0000 7f73f6200640 1 -- 172.21.15.67:0/3734158741 >> 172.21.15.67:0/3734158741 conn(0x7f73f00fd5b0 msgr2=0x7f73f00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.736+0000 7f73f6200640 1 -- 172.21.15.67:0/3734158741 shutdown_connections 2024-09-10T22:58:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.736+0000 7f73f6200640 1 -- 172.21.15.67:0/3734158741 wait complete. 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.736+0000 7f73f6200640 1 Processor -- start 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 -- start start 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 0x7f73f0101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f0102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112ae0 con 0x7f73f010cf70 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112960 con 0x7f73f0104430 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112c60 con 0x7f73f0103b10 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f0102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73efe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 0x7f73f0101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51052/0 (socket says 172.21.15.67:51052) 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73efe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 0x7f73f0101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:40216/0 (socket says 172.21.15.67:40216) 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.737+0000 7f73f4e00640 1 -- 172.21.15.67:0/3804703778 learned_addr learned my addr 172.21.15.67:0/3804703778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 -- 172.21.15.67:0/3804703778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 msgr2=0x7f73f0101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 0x7f73f0101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 -- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 msgr2=0x7f73f0102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f0102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 -- 172.21.15.67:0/3804703778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f01a7370 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73f4e00640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e400ea60 tx=0x7f73e400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.738+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e400cde0 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.739+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f01a7660 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.739+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73e40035f0 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.739+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4010690 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.739+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01a7bf0 con 0x7f73f0103b10 2024-09-10T22:58:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.740+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f0103ef0 con 0x7f73f0103b10 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.743+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f73e4004000 con 0x7f73f0103b10 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.744+0000 7f73ed600640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 0x7f73d80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.744+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f73e40e6480 con 0x7f73f0103b10 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.744+0000 7f73efe00640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 0x7f73d80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.745+0000 7f73efe00640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 0x7f73d80b79a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73f01032b0 tx=0x7f73dc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.745+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f73e40acd60 con 0x7f73f0103b10 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/633593148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:22 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/893813968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:58:22.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/633593148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:58:22.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:22 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/893813968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:58:22.910 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/633593148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:58:22.910 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.910 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.910 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:22.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:22 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/893813968' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:58:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.946+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f73f0102ac0 con 0x7f73f0103b10 2024-09-10T22:58:22.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.947+0000 7f73ed600640 1 -- 172.21.15.67:0/3804703778 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73e40ac660 con 0x7f73f0103b10 2024-09-10T22:58:22.949 INFO:teuthology.orchestra.run.smithi067.stdout:188978561094 2024-09-10T22:58:22.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.949+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 msgr2=0x7f73d80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.949+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 0x7f73d80b79a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73f01032b0 tx=0x7f73dc005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 msgr2=0x7f73f01017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e400ea60 tx=0x7f73e400ef20 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 shutdown_connections 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f73d80b54e0 0x7f73d80b79a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73f010cf70 0x7f73f0102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f0104430 0x7f73f0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 --2- 172.21.15.67:0/3804703778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f0103b10 0x7f73f01017c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 >> 172.21.15.67:0/3804703778 conn(0x7f73f00fd5b0 msgr2=0x7f73f010f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 shutdown_connections 2024-09-10T22:58:22.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.950+0000 7f73f6200640 1 -- 172.21.15.67:0/3804703778 wait complete. 2024-09-10T22:58:23.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397563 got 210453397566 for osd.7 2024-09-10T22:58:23.272 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:23.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117076 got 115964117079 for osd.3 2024-09-10T22:58:23.626 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.993+0000 7f2280800640 1 -- 172.21.15.67:0/3403570841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f227810b330 msgr2=0x7f2278111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.993+0000 7f2280800640 1 --2- 172.21.15.67:0/3403570841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f227810b330 0x7f2278111b30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2274009ca0 tx=0x7f227402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 -- 172.21.15.67:0/3403570841 shutdown_connections 2024-09-10T22:58:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 --2- 172.21.15.67:0/3403570841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f227810b330 0x7f2278111b30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 --2- 172.21.15.67:0/3403570841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810a9b0 0x7f227810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 --2- 172.21.15.67:0/3403570841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2278104fa0 0x7f2278105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 -- 172.21.15.67:0/3403570841 >> 172.21.15.67:0/3403570841 conn(0x7f2278100a50 msgr2=0x7f2278102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 -- 172.21.15.67:0/3403570841 shutdown_connections 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.994+0000 7f2280800640 1 -- 172.21.15.67:0/3403570841 wait complete. 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.995+0000 7f2280800640 1 Processor -- start 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.995+0000 7f2280800640 1 -- start start 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f227810a9b0 0x7f227819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810b330 0x7f22781a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2278114e60 con 0x7f2278104fa0 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2278114ce0 con 0x7f227810a9b0 2024-09-10T22:58:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f2280800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2278114fe0 con 0x7f227810b330 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f227de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f227de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51684/0 (socket says 172.21.15.67:51684) 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f227de00640 1 -- 172.21.15.67:0/893813968 learned_addr learned my addr 172.21.15.67:0/893813968 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f227d400640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f227810a9b0 0x7f227819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.996+0000 7f227e800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810b330 0x7f22781a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 -- 172.21.15.67:0/893813968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810b330 msgr2=0x7f22781a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810b330 0x7f22781a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 -- 172.21.15.67:0/893813968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f227810a9b0 msgr2=0x7f227819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f227810a9b0 0x7f227819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 -- 172.21.15.67:0/893813968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22781a1420 con 0x7f2278104fa0 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f227de00640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e9f0 tx=0x7f226c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c00cd80 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f226c0108c0 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0187c0 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22781a1710 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.997+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781a7dd0 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:21.999+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2278105380 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.001+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f226c010430 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.002+0000 7f2272a00640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 0x7f22600b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.002+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f226c0e69d0 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.003+0000 7f227d400640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 0x7f22600b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.003+0000 7f227d400640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 0x7f22600b7900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2278107380 tx=0x7f2268005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.004+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f226c0acc60 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.231+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f227802e5c0 con 0x7f2278104fa0 2024-09-10T22:58:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.232+0000 7f2272a00640 1 -- 172.21.15.67:0/893813968 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f226c0b1690 con 0x7f2278104fa0 2024-09-10T22:58:23.684 INFO:teuthology.orchestra.run.smithi067.stdout:287762808877 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.234+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 msgr2=0x7f22600b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.234+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 0x7f22600b7900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2278107380 tx=0x7f2268005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.234+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 msgr2=0x7f227819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.234+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e9f0 tx=0x7f226c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 shutdown_connections 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f22600b5440 0x7f22600b7900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f227810b330 0x7f22781a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f227810a9b0 0x7f227819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 --2- 172.21.15.67:0/893813968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2278104fa0 0x7f227819b370 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 >> 172.21.15.67:0/893813968 conn(0x7f2278100a50 msgr2=0x7f2278102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 shutdown_connections 2024-09-10T22:58:23.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:22.235+0000 7f2280800640 1 -- 172.21.15.67:0/893813968 wait complete. 2024-09-10T22:58:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:23 smithi145 ceph-mon[34397]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:23 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3804703778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:58:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:23 smithi092 ceph-mon[34223]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:23 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3804703778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:58:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:23 smithi067 ceph-mon[30939]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:23 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3804703778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 -- 172.21.15.67:0/1399795110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 msgr2=0x7fd1bc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 --2- 172.21.15.67:0/1399795110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc1031a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 -- 172.21.15.67:0/1399795110 shutdown_connections 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 --2- 172.21.15.67:0/1399795110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc109150 0x7fd1bc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 --2- 172.21.15.67:0/1399795110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 --2- 172.21.15.67:0/1399795110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc1031a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 -- 172.21.15.67:0/1399795110 >> 172.21.15.67:0/1399795110 conn(0x7fd1bc0fe830 msgr2=0x7fd1bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 -- 172.21.15.67:0/1399795110 shutdown_connections 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.592+0000 7fd1c2800640 1 -- 172.21.15.67:0/1399795110 wait complete. 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.593+0000 7fd1c2800640 1 Processor -- start 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.593+0000 7fd1c2800640 1 -- start start 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.593+0000 7fd1c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1c2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1c2800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc109150 0x7fd1bc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1c2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc112cd0 con 0x7fd1bc102dc0 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1c2800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc112b50 con 0x7fd1bc109150 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1c2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc112e50 con 0x7fd1bc1087d0 2024-09-10T22:58:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51728/0 (socket says 172.21.15.67:51728) 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1bb400640 1 -- 172.21.15.67:0/513296292 learned_addr learned my addr 172.21.15.67:0/513296292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1bbe00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc109150 0x7fd1bc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 -- 172.21.15.67:0/513296292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc109150 msgr2=0x7fd1bc1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc109150 0x7fd1bc1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 -- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 msgr2=0x7fd1bc19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1baa00640 1 -- 172.21.15.67:0/513296292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc1a14d0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.594+0000 7fd1bb400640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1baa00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc19b960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e9f0 tx=0x7fd1a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800ccc0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1a80108c0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80187e0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc1a17c0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.595+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a7e80 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.596+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc1031a0 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.599+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fd1a8010430 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.601+0000 7fd1b3e00640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 0x7fd1a00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.601+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fd1a80e6790 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.601+0000 7fd1bb400640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 0x7fd1a00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.602+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd1a80aca20 con 0x7fd1bc1087d0 2024-09-10T22:58:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.602+0000 7fd1bb400640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 0x7fd1a00b7880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0096c0 tx=0x7fd1ac005e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.802+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd1bc19c550 con 0x7fd1bc1087d0 2024-09-10T22:58:24.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.803+0000 7fd1b3e00640 1 -- 172.21.15.67:0/513296292 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1a80b1450 con 0x7fd1bc1087d0 2024-09-10T22:58:24.805 INFO:teuthology.orchestra.run.smithi067.stdout:240518168635 2024-09-10T22:58:24.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 msgr2=0x7fd1a00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 0x7fd1a00b7880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0096c0 tx=0x7fd1ac005e60 comp rx=0 tx=0).stop 2024-09-10T22:58:24.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 msgr2=0x7fd1bc19b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc19b960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e9f0 tx=0x7fd1a800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 shutdown_connections 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd1a00b53c0 0x7fd1a00b7880 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1bc109150 0x7fd1bc1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1bc1087d0 0x7fd1bc19b960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 --2- 172.21.15.67:0/513296292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1bc102dc0 0x7fd1bc19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.805+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 >> 172.21.15.67:0/513296292 conn(0x7fd1bc0fe830 msgr2=0x7fd1bc104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.806+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 shutdown_connections 2024-09-10T22:58:24.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.806+0000 7fd1c2800640 1 -- 172.21.15.67:0/513296292 wait complete. 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 -- 172.21.15.67:0/1611759724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 msgr2=0x7f48dc102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 --2- 172.21.15.67:0/1611759724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc102ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009900 tx=0x7f48cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 -- 172.21.15.67:0/1611759724 shutdown_connections 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 --2- 172.21.15.67:0/1611759724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 --2- 172.21.15.67:0/1611759724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 --2- 172.21.15.67:0/1611759724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc102ff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 -- 172.21.15.67:0/1611759724 >> 172.21.15.67:0/1611759724 conn(0x7f48dc0fe6c0 msgr2=0x7f48dc100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 -- 172.21.15.67:0/1611759724 shutdown_connections 2024-09-10T22:58:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.926+0000 7f48e1a00640 1 -- 172.21.15.67:0/1611759724 wait complete. 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.927+0000 7f48e1a00640 1 Processor -- start 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.927+0000 7f48e1a00640 1 -- start start 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc112b60 con 0x7f48dc108fa0 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1129e0 con 0x7f48dc102c10 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc112ce0 con 0x7f48dc108620 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51756/0 (socket says 172.21.15.67:51756) 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 -- 172.21.15.67:0/568569015 learned_addr learned my addr 172.21.15.67:0/568569015 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48dbe00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48db400640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 -- 172.21.15.67:0/568569015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 msgr2=0x7f48dc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 -- 172.21.15.67:0/568569015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 msgr2=0x7f48dc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48e1000640 1 -- 172.21.15.67:0/568569015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc1a7390 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.928+0000 7f48db400640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48e1000640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f48d400eb10 tx=0x7f48d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4004b20 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48d4004cc0 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc1a7680 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4010690 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.929+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a7b70 con 0x7f48dc108fa0 2024-09-10T22:58:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.930+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc102ff0 con 0x7f48dc108fa0 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.932+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f48d4005020 con 0x7f48dc108fa0 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.934+0000 7f48d8c00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 0x7f48c00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.934+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f48d40e68b0 con 0x7f48dc108fa0 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.934+0000 7f48dbe00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 0x7f48c00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.935+0000 7f48dbe00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 0x7f48c00b7920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48cc006010 tx=0x7f48cc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:24.938+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f48d40ad190 con 0x7f48dc108fa0 2024-09-10T22:58:25.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.137+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f48dc10a7f0 con 0x7f48dc108fa0 2024-09-10T22:58:25.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.137+0000 7f48d8c00640 1 -- 172.21.15.67:0/568569015 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48d40aca90 con 0x7f48dc108fa0 2024-09-10T22:58:25.139 INFO:teuthology.orchestra.run.smithi067.stdout:60129542245 2024-09-10T22:58:25.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.139+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 msgr2=0x7f48c00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:25.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.139+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 0x7f48c00b7920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48cc006010 tx=0x7f48cc005e80 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.139+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 msgr2=0x7f48dc075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.139+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f48d400eb10 tx=0x7f48d400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 shutdown_connections 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f48c00b5460 0x7f48c00b7920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc108fa0 0x7f48dc075f20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48dc108620 0x7f48dc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 --2- 172.21.15.67:0/568569015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48dc102c10 0x7f48dc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 >> 172.21.15.67:0/568569015 conn(0x7f48dc0fe6c0 msgr2=0x7f48dc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 shutdown_connections 2024-09-10T22:58:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:25.140+0000 7f48e1a00640 1 -- 172.21.15.67:0/568569015 wait complete. 2024-09-10T22:58:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:25 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/513296292' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:58:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:25 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/513296292' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:58:25.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:25.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:25.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:25 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/513296292' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:58:26.213 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612646 got 313532612648 for osd.11 2024-09-10T22:58:26.213 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:26 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/568569015' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:58:26.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:26 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/568569015' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:58:26.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:26 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/568569015' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:58:26.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542240 got 60129542245 for osd.1 2024-09-10T22:58:26.617 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:26.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757321 got 163208757323 for osd.5 2024-09-10T22:58:26.743 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:27.481 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:27 smithi067 ceph-mon[30939]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:27 smithi092 ceph-mon[34223]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:27 smithi145 ceph-mon[34397]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:58:28.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:28 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1512388588' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:58:28.538 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:28 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1512388588' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:58:28.578 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378717 got 81604378719 for osd.2 2024-09-10T22:58:28.578 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:28.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:28 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1512388588' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:58:29.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920846 got 141733920849 for osd.4 2024-09-10T22:58:29.257 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:29.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561091 got 188978561094 for osd.6 2024-09-10T22:58:29.260 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:29.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:29 smithi092 ceph-mon[34223]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:29.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:29 smithi067 ceph-mon[30939]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:29.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:29 smithi145 ceph-mon[34397]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:29.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808874 got 287762808877 for osd.10 2024-09-10T22:58:29.659 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 -- 172.21.15.67:0/3582177316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5820109150 msgr2=0x7f582010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 --2- 172.21.15.67:0/3582177316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5820109150 0x7f582010f950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5818009ca0 tx=0x7f581802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 -- 172.21.15.67:0/3582177316 shutdown_connections 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 --2- 172.21.15.67:0/3582177316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5820109150 0x7f582010f950 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 --2- 172.21.15.67:0/3582177316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58201087d0 0x7f5820108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 --2- 172.21.15.67:0/3582177316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 0x7f58201031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 -- 172.21.15.67:0/3582177316 >> 172.21.15.67:0/3582177316 conn(0x7f58200fe830 msgr2=0x7f5820100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 -- 172.21.15.67:0/3582177316 shutdown_connections 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.268+0000 7f5826c00640 1 -- 172.21.15.67:0/3582177316 wait complete. 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 Processor -- start 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 -- start start 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 0x7f582019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5820109150 0x7f58201a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5820112d10 con 0x7f58201087d0 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5820112b90 con 0x7f5820102dc0 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.269+0000 7f5826c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5820112e90 con 0x7f5820109150 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51766/0 (socket says 172.21.15.67:51766) 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 -- 172.21.15.67:0/1512388588 learned_addr learned my addr 172.21.15.67:0/1512388588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f581fe00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 0x7f582019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5824c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5820109150 0x7f58201a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 -- 172.21.15.67:0/1512388588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5820109150 msgr2=0x7f58201a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5820109150 0x7f58201a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 -- 172.21.15.67:0/1512388588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 msgr2=0x7f582019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 0x7f582019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 -- 172.21.15.67:0/1512388588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58201a1430 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.270+0000 7f5817400640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f580800d950 tx=0x7f580800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.271+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808017070 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.271+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5808009c30 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.271+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800f620 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.271+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58201a1720 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.271+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58201a7de0 con 0x7f58201087d0 2024-09-10T22:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.272+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58201031a0 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.274+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f5808010460 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.277+0000 7f581d600640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 0x7f58040b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.277+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f58080e5cf0 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.277+0000 7f581fe00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 0x7f58040b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.278+0000 7f581fe00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 0x7f58040b7900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f581000bbe0 tx=0x7f581000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.281+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5808009de0 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.479+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f582002e5c0 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.479+0000 7f581d600640 1 -- 172.21.15.67:0/1512388588 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58080abfe0 con 0x7f58201087d0 2024-09-10T22:58:29.809 INFO:teuthology.orchestra.run.smithi067.stdout:266287972404 2024-09-10T22:58:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.481+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 msgr2=0x7f58040b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 0x7f58040b7900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f581000bbe0 tx=0x7f581000b390 comp rx=0 tx=0).stop 2024-09-10T22:58:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 msgr2=0x7f582019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f580800d950 tx=0x7f580800de10 comp rx=0 tx=0).stop 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 shutdown_connections 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58040b5440 0x7f58040b7900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5820109150 0x7f58201a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58201087d0 0x7f582019b8c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 --2- 172.21.15.67:0/1512388588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5820102dc0 0x7f582019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 >> 172.21.15.67:0/1512388588 conn(0x7f58200fe830 msgr2=0x7f5820104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 shutdown_connections 2024-09-10T22:58:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:27.482+0000 7f5826c00640 1 -- 172.21.15.67:0/1512388588 wait complete. 2024-09-10T22:58:30.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972401 got 266287972404 for osd.9 2024-09-10T22:58:30.508 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:30.907 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168633 got 240518168635 for osd.8 2024-09-10T22:58:30.907 DEBUG:teuthology.parallel:result is None 2024-09-10T22:58:30.908 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:58:30.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:58:31.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:31 smithi067 ceph-mon[30939]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:31.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:31 smithi092 ceph-mon[34223]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:31 smithi145 ceph-mon[34397]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:31.971 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:32.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 -- 172.21.15.67:0/244828913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 msgr2=0x7f005810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:32.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 --2- 172.21.15.67:0/244828913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005810adf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0040009900 tx=0x7f004002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:32.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 -- 172.21.15.67:0/244828913 shutdown_connections 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 --2- 172.21.15.67:0/244828913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 0x7f0058111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 --2- 172.21.15.67:0/244828913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005810adf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 --2- 172.21.15.67:0/244828913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f0058105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 -- 172.21.15.67:0/244828913 >> 172.21.15.67:0/244828913 conn(0x7f0058100a50 msgr2=0x7f0058102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 -- 172.21.15.67:0/244828913 shutdown_connections 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.591+0000 7f005e400640 1 -- 172.21.15.67:0/244828913 wait complete. 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.592+0000 7f005e400640 1 Processor -- start 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.592+0000 7f005e400640 1 -- start start 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f005819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 0x7f00581a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114de0 con 0x7f005810a9b0 2024-09-10T22:58:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114c60 con 0x7f0058104fa0 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f005e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114f60 con 0x7f005810b330 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f004ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f004ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60358/0 (socket says 172.21.15.67:60358) 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f004ea00640 1 -- 172.21.15.67:0/1958501116 learned_addr learned my addr 172.21.15.67:0/1958501116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f0057e00640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 0x7f00581a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.593+0000 7f0057400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f005819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 -- 172.21.15.67:0/1958501116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 msgr2=0x7f00581a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 0x7f00581a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 -- 172.21.15.67:0/1958501116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 msgr2=0x7f005819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f005819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 -- 172.21.15.67:0/1958501116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00581a1460 con 0x7f005810a9b0 2024-09-10T22:58:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f0057400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f005819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f004ea00640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0040009230 tx=0x7f00400319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004003f030 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00400043d0 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00581a16f0 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040035070 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.594+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581a7e10 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.596+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0058105380 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.598+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f004003b8c0 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.599+0000 7f0054c00640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 0x7f003c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.599+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f0040106920 con 0x7f005810a9b0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.599+0000 7f0057400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 0x7f003c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.600+0000 7f0057400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 0x7f003c0b7880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0048009c20 tx=0x7f0048005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:32.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.603+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f00400cd200 con 0x7f005810a9b0 2024-09-10T22:58:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.816+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0058063590 con 0x7f003c0b53c0 2024-09-10T22:58:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.823+0000 7f0054c00640 1 -- 172.21.15.67:0/1958501116 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340624 (secure 0 0 0) 0x7f0058063590 con 0x7f003c0b53c0 2024-09-10T22:58:32.829 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:32.831 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:58:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 msgr2=0x7f003c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 0x7f003c0b7880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0048009c20 tx=0x7f0048005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 msgr2=0x7f005819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0040009230 tx=0x7f00400319a0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 shutdown_connections 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f003c0b53c0 0x7f003c0b7880 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f005810b330 0x7f00581a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f005810a9b0 0x7f005819b8f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 --2- 172.21.15.67:0/1958501116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0058104fa0 0x7f005819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.828+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 >> 172.21.15.67:0/1958501116 conn(0x7f0058100a50 msgr2=0x7f0058102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.829+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 shutdown_connections 2024-09-10T22:58:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:32.829+0000 7f005e400640 1 -- 172.21.15.67:0/1958501116 wait complete. 2024-09-10T22:58:33.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:33.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:58:33.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-10T22:58:33.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:58:33.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:33 smithi092 ceph-mon[34223]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:33.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:33 smithi067 ceph-mon[30939]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:58:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-10T22:58:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:58:33.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-09-10T22:58:33.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:33 smithi145 ceph-mon[34397]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:33.649 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-10T22:58:32.092641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656675,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":571,"num_read_kb":387,"num_write":293,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356816,"kb_used_data":11672,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585008,"statfs":{"total":1151940427776,"available":1151575048192,"internally_reserved":0,"allocated":11952128,"data_stored":7933223,"data_compressed":391371,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003720"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485634+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:33.485634+0000","last_peered":"2024-09-10T22:57:33.485634+0000","last_clean":"2024-09-10T22:57:33.485634+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:33.485634+0000","last_undegraded":"2024-09-10T22:57:33.485634+0000","last_fullsized":"2024-09-10T22:57:33.485634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483361+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:57:33.483361+0000","last_peered":"2024-09-10T22:57:33.483361+0000","last_clean":"2024-09-10T22:57:33.483361+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:57:33.483361+0000","last_undegraded":"2024-09-10T22:57:33.483361+0000","last_fullsized":"2024-09-10T22:57:33.483361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488147+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:57:33.488147+0000","last_peered":"2024-09-10T22:57:33.488147+0000","last_clean":"2024-09-10T22:57:33.488147+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:57:33.488147+0000","last_undegraded":"2024-09-10T22:57:33.488147+0000","last_fullsized":"2024-09-10T22:57:33.488147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483555+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:57:33.483555+0000","last_peered":"2024-09-10T22:57:33.483555+0000","last_clean":"2024-09-10T22:57:33.483555+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:57:33.483555+0000","last_undegraded":"2024-09-10T22:57:33.483555+0000","last_fullsized":"2024-09-10T22:57:33.483555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492613+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:57:33.492613+0000","last_peered":"2024-09-10T22:57:33.492613+0000","last_clean":"2024-09-10T22:57:33.492613+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:57:33.492613+0000","last_undegraded":"2024-09-10T22:57:33.492613+0000","last_fullsized":"2024-09-10T22:57:33.492613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933763+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:57:33.933763+0000","last_peered":"2024-09-10T22:57:33.933763+0000","last_clean":"2024-09-10T22:57:33.933763+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:57:33.933763+0000","last_undegraded":"2024-09-10T22:57:33.933763+0000","last_fullsized":"2024-09-10T22:57:33.933763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495695+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:33.495695+0000","last_peered":"2024-09-10T22:57:33.495695+0000","last_clean":"2024-09-10T22:57:33.495695+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:33.495695+0000","last_undegraded":"2024-09-10T22:57:33.495695+0000","last_fullsized":"2024-09-10T22:57:33.495695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484689+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:57:33.484689+0000","last_peered":"2024-09-10T22:57:33.484689+0000","last_clean":"2024-09-10T22:57:33.484689+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:57:33.484689+0000","last_undegraded":"2024-09-10T22:57:33.484689+0000","last_fullsized":"2024-09-10T22:57:33.484689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933235+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:57:33.933235+0000","last_peered":"2024-09-10T22:57:33.933235+0000","last_clean":"2024-09-10T22:57:33.933235+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:57:33.933235+0000","last_undegraded":"2024-09-10T22:57:33.933235+0000","last_fullsized":"2024-09-10T22:57:33.933235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.499755+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:57:33.499755+0000","last_peered":"2024-09-10T22:57:33.499755+0000","last_clean":"2024-09-10T22:57:33.499755+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:57:33.499755+0000","last_undegraded":"2024-09-10T22:57:33.499755+0000","last_fullsized":"2024-09-10T22:57:33.499755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.637598+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:57:33.637598+0000","last_peered":"2024-09-10T22:57:33.637598+0000","last_clean":"2024-09-10T22:57:33.637598+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:57:33.637598+0000","last_undegraded":"2024-09-10T22:57:33.637598+0000","last_fullsized":"2024-09-10T22:57:33.637598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933169+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:57:33.933169+0000","last_peered":"2024-09-10T22:57:33.933169+0000","last_clean":"2024-09-10T22:57:33.933169+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:57:33.933169+0000","last_undegraded":"2024-09-10T22:57:33.933169+0000","last_fullsized":"2024-09-10T22:57:33.933169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489454+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:57:33.489454+0000","last_peered":"2024-09-10T22:57:33.489454+0000","last_clean":"2024-09-10T22:57:33.489454+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:57:33.489454+0000","last_undegraded":"2024-09-10T22:57:33.489454+0000","last_fullsized":"2024-09-10T22:57:33.489454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494981+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:33.494981+0000","last_peered":"2024-09-10T22:57:33.494981+0000","last_clean":"2024-09-10T22:57:33.494981+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:33.494981+0000","last_undegraded":"2024-09-10T22:57:33.494981+0000","last_fullsized":"2024-09-10T22:57:33.494981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933680+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:57:33.933680+0000","last_peered":"2024-09-10T22:57:33.933680+0000","last_clean":"2024-09-10T22:57:33.933680+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:57:33.933680+0000","last_undegraded":"2024-09-10T22:57:33.933680+0000","last_fullsized":"2024-09-10T22:57:33.933680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483072+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:57:33.483072+0000","last_peered":"2024-09-10T22:57:33.483072+0000","last_clean":"2024-09-10T22:57:33.483072+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:57:33.483072+0000","last_undegraded":"2024-09-10T22:57:33.483072+0000","last_fullsized":"2024-09-10T22:57:33.483072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488718+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:57:33.488718+0000","last_peered":"2024-09-10T22:57:33.488718+0000","last_clean":"2024-09-10T22:57:33.488718+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:57:33.488718+0000","last_undegraded":"2024-09-10T22:57:33.488718+0000","last_fullsized":"2024-09-10T22:57:33.488718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933688+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:57:33.933688+0000","last_peered":"2024-09-10T22:57:33.933688+0000","last_clean":"2024-09-10T22:57:33.933688+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:57:33.933688+0000","last_undegraded":"2024-09-10T22:57:33.933688+0000","last_fullsized":"2024-09-10T22:57:33.933688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489640+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:57:33.489640+0000","last_peered":"2024-09-10T22:57:33.489640+0000","last_clean":"2024-09-10T22:57:33.489640+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:57:33.489640+0000","last_undegraded":"2024-09-10T22:57:33.489640+0000","last_fullsized":"2024-09-10T22:57:33.489640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483047+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:57:33.483047+0000","last_peered":"2024-09-10T22:57:33.483047+0000","last_clean":"2024-09-10T22:57:33.483047+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:57:33.483047+0000","last_undegraded":"2024-09-10T22:57:33.483047+0000","last_fullsized":"2024-09-10T22:57:33.483047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485365+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:57:33.485365+0000","last_peered":"2024-09-10T22:57:33.485365+0000","last_clean":"2024-09-10T22:57:33.485365+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:57:33.485365+0000","last_undegraded":"2024-09-10T22:57:33.485365+0000","last_fullsized":"2024-09-10T22:57:33.485365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494037+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:57:33.494037+0000","last_peered":"2024-09-10T22:57:33.494037+0000","last_clean":"2024-09-10T22:57:33.494037+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:57:33.494037+0000","last_undegraded":"2024-09-10T22:57:33.494037+0000","last_fullsized":"2024-09-10T22:57:33.494037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484404+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:57:33.484404+0000","last_peered":"2024-09-10T22:57:33.484404+0000","last_clean":"2024-09-10T22:57:33.484404+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:57:33.484404+0000","last_undegraded":"2024-09-10T22:57:33.484404+0000","last_fullsized":"2024-09-10T22:57:33.484404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933699+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:57:33.933699+0000","last_peered":"2024-09-10T22:57:33.933699+0000","last_clean":"2024-09-10T22:57:33.933699+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:57:33.933699+0000","last_undegraded":"2024-09-10T22:57:33.933699+0000","last_fullsized":"2024-09-10T22:57:33.933699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492482+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:57:33.492482+0000","last_peered":"2024-09-10T22:57:33.492482+0000","last_clean":"2024-09-10T22:57:33.492482+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:57:33.492482+0000","last_undegraded":"2024-09-10T22:57:33.492482+0000","last_fullsized":"2024-09-10T22:57:33.492482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934332+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:33.934332+0000","last_peered":"2024-09-10T22:57:33.934332+0000","last_clean":"2024-09-10T22:57:33.934332+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:33.934332+0000","last_undegraded":"2024-09-10T22:57:33.934332+0000","last_fullsized":"2024-09-10T22:57:33.934332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488401+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:57:33.488401+0000","last_peered":"2024-09-10T22:57:33.488401+0000","last_clean":"2024-09-10T22:57:33.488401+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:57:33.488401+0000","last_undegraded":"2024-09-10T22:57:33.488401+0000","last_fullsized":"2024-09-10T22:57:33.488401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483493+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:57:33.483493+0000","last_peered":"2024-09-10T22:57:33.483493+0000","last_clean":"2024-09-10T22:57:33.483493+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:57:33.483493+0000","last_undegraded":"2024-09-10T22:57:33.483493+0000","last_fullsized":"2024-09-10T22:57:33.483493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485444+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:33.485444+0000","last_peered":"2024-09-10T22:57:33.485444+0000","last_clean":"2024-09-10T22:57:33.485444+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:33.485444+0000","last_undegraded":"2024-09-10T22:57:33.485444+0000","last_fullsized":"2024-09-10T22:57:33.485444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482762+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:57:33.482762+0000","last_peered":"2024-09-10T22:57:33.482762+0000","last_clean":"2024-09-10T22:57:33.482762+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:57:33.482762+0000","last_undegraded":"2024-09-10T22:57:33.482762+0000","last_fullsized":"2024-09-10T22:57:33.482762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636096+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:57:33.636096+0000","last_peered":"2024-09-10T22:57:33.636096+0000","last_clean":"2024-09-10T22:57:33.636096+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:57:33.636096+0000","last_undegraded":"2024-09-10T22:57:33.636096+0000","last_fullsized":"2024-09-10T22:57:33.636096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495769+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:33.495769+0000","last_peered":"2024-09-10T22:57:33.495769+0000","last_clean":"2024-09-10T22:57:33.495769+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:33.495769+0000","last_undegraded":"2024-09-10T22:57:33.495769+0000","last_fullsized":"2024-09-10T22:57:33.495769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487731+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:33.487731+0000","last_peered":"2024-09-10T22:57:33.487731+0000","last_clean":"2024-09-10T22:57:33.487731+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:33.487731+0000","last_undegraded":"2024-09-10T22:57:33.487731+0000","last_fullsized":"2024-09-10T22:57:33.487731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489169+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:57:33.489169+0000","last_peered":"2024-09-10T22:57:33.489169+0000","last_clean":"2024-09-10T22:57:33.489169+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:57:33.489169+0000","last_undegraded":"2024-09-10T22:57:33.489169+0000","last_fullsized":"2024-09-10T22:57:33.489169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485206+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:57:33.485206+0000","last_peered":"2024-09-10T22:57:33.485206+0000","last_clean":"2024-09-10T22:57:33.485206+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:57:33.485206+0000","last_undegraded":"2024-09-10T22:57:33.485206+0000","last_fullsized":"2024-09-10T22:57:33.485206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493448+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:57:33.493448+0000","last_peered":"2024-09-10T22:57:33.493448+0000","last_clean":"2024-09-10T22:57:33.493448+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:57:33.493448+0000","last_undegraded":"2024-09-10T22:57:33.493448+0000","last_fullsized":"2024-09-10T22:57:33.493448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490145+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:57:33.490145+0000","last_peered":"2024-09-10T22:57:33.490145+0000","last_clean":"2024-09-10T22:57:33.490145+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:57:33.490145+0000","last_undegraded":"2024-09-10T22:57:33.490145+0000","last_fullsized":"2024-09-10T22:57:33.490145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489648+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:57:33.489648+0000","last_peered":"2024-09-10T22:57:33.489648+0000","last_clean":"2024-09-10T22:57:33.489648+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:57:33.489648+0000","last_undegraded":"2024-09-10T22:57:33.489648+0000","last_fullsized":"2024-09-10T22:57:33.489648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635906+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:57:33.635906+0000","last_peered":"2024-09-10T22:57:33.635906+0000","last_clean":"2024-09-10T22:57:33.635906+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:57:33.635906+0000","last_undegraded":"2024-09-10T22:57:33.635906+0000","last_fullsized":"2024-09-10T22:57:33.635906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484298+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:57:33.484298+0000","last_peered":"2024-09-10T22:57:33.484298+0000","last_clean":"2024-09-10T22:57:33.484298+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:57:33.484298+0000","last_undegraded":"2024-09-10T22:57:33.484298+0000","last_fullsized":"2024-09-10T22:57:33.484298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:28.587447+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:58:28.587447+0000","last_peered":"2024-09-10T22:58:28.587447+0000","last_clean":"2024-09-10T22:58:28.587447+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:58:28.587447+0000","last_undegraded":"2024-09-10T22:58:28.587447+0000","last_fullsized":"2024-09-10T22:58:28.587447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636145+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:57:33.636145+0000","last_peered":"2024-09-10T22:57:33.636145+0000","last_clean":"2024-09-10T22:57:33.636145+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:57:33.636145+0000","last_undegraded":"2024-09-10T22:57:33.636145+0000","last_fullsized":"2024-09-10T22:57:33.636145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635709+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:57:33.635709+0000","last_peered":"2024-09-10T22:57:33.635709+0000","last_clean":"2024-09-10T22:57:33.635709+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:57:33.635709+0000","last_undegraded":"2024-09-10T22:57:33.635709+0000","last_fullsized":"2024-09-10T22:57:33.635709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487562+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:57:33.487562+0000","last_peered":"2024-09-10T22:57:33.487562+0000","last_clean":"2024-09-10T22:57:33.487562+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:57:33.487562+0000","last_undegraded":"2024-09-10T22:57:33.487562+0000","last_fullsized":"2024-09-10T22:57:33.487562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484718+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:57:33.484718+0000","last_peered":"2024-09-10T22:57:33.484718+0000","last_clean":"2024-09-10T22:57:33.484718+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:57:33.484718+0000","last_undegraded":"2024-09-10T22:57:33.484718+0000","last_fullsized":"2024-09-10T22:57:33.484718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489346+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:57:33.489346+0000","last_peered":"2024-09-10T22:57:33.489346+0000","last_clean":"2024-09-10T22:57:33.489346+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:57:33.489346+0000","last_undegraded":"2024-09-10T22:57:33.489346+0000","last_fullsized":"2024-09-10T22:57:33.489346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934203+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:33.934203+0000","last_peered":"2024-09-10T22:57:33.934203+0000","last_clean":"2024-09-10T22:57:33.934203+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:33.934203+0000","last_undegraded":"2024-09-10T22:57:33.934203+0000","last_fullsized":"2024-09-10T22:57:33.934203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487351+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:57:33.487351+0000","last_peered":"2024-09-10T22:57:33.487351+0000","last_clean":"2024-09-10T22:57:33.487351+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:57:33.487351+0000","last_undegraded":"2024-09-10T22:57:33.487351+0000","last_fullsized":"2024-09-10T22:57:33.487351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487434+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:57:33.487434+0000","last_peered":"2024-09-10T22:57:33.487434+0000","last_clean":"2024-09-10T22:57:33.487434+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:57:33.487434+0000","last_undegraded":"2024-09-10T22:57:33.487434+0000","last_fullsized":"2024-09-10T22:57:33.487434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487965+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:57:33.487965+0000","last_peered":"2024-09-10T22:57:33.487965+0000","last_clean":"2024-09-10T22:57:33.487965+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:57:33.487965+0000","last_undegraded":"2024-09-10T22:57:33.487965+0000","last_fullsized":"2024-09-10T22:57:33.487965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489704+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:57:33.489704+0000","last_peered":"2024-09-10T22:57:33.489704+0000","last_clean":"2024-09-10T22:57:33.489704+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:57:33.489704+0000","last_undegraded":"2024-09-10T22:57:33.489704+0000","last_fullsized":"2024-09-10T22:57:33.489704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484515+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:57:33.484515+0000","last_peered":"2024-09-10T22:57:33.484515+0000","last_clean":"2024-09-10T22:57:33.484515+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:57:33.484515+0000","last_undegraded":"2024-09-10T22:57:33.484515+0000","last_fullsized":"2024-09-10T22:57:33.484515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495905+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:57:33.495905+0000","last_peered":"2024-09-10T22:57:33.495905+0000","last_clean":"2024-09-10T22:57:33.495905+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:57:33.495905+0000","last_undegraded":"2024-09-10T22:57:33.495905+0000","last_fullsized":"2024-09-10T22:57:33.495905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485826+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:57:33.485826+0000","last_peered":"2024-09-10T22:57:33.485826+0000","last_clean":"2024-09-10T22:57:33.485826+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:57:33.485826+0000","last_undegraded":"2024-09-10T22:57:33.485826+0000","last_fullsized":"2024-09-10T22:57:33.485826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492168+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:57:33.492168+0000","last_peered":"2024-09-10T22:57:33.492168+0000","last_clean":"2024-09-10T22:57:33.492168+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:57:33.492168+0000","last_undegraded":"2024-09-10T22:57:33.492168+0000","last_fullsized":"2024-09-10T22:57:33.492168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487627+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:57:33.487627+0000","last_peered":"2024-09-10T22:57:33.487627+0000","last_clean":"2024-09-10T22:57:33.487627+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:57:33.487627+0000","last_undegraded":"2024-09-10T22:57:33.487627+0000","last_fullsized":"2024-09-10T22:57:33.487627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636233+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:57:33.636233+0000","last_peered":"2024-09-10T22:57:33.636233+0000","last_clean":"2024-09-10T22:57:33.636233+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:57:33.636233+0000","last_undegraded":"2024-09-10T22:57:33.636233+0000","last_fullsized":"2024-09-10T22:57:33.636233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483887+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:57:33.483887+0000","last_peered":"2024-09-10T22:57:33.483887+0000","last_clean":"2024-09-10T22:57:33.483887+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:57:33.483887+0000","last_undegraded":"2024-09-10T22:57:33.483887+0000","last_fullsized":"2024-09-10T22:57:33.483887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489318+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:57:33.489318+0000","last_peered":"2024-09-10T22:57:33.489318+0000","last_clean":"2024-09-10T22:57:33.489318+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:57:33.489318+0000","last_undegraded":"2024-09-10T22:57:33.489318+0000","last_fullsized":"2024-09-10T22:57:33.489318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495716+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:33.495716+0000","last_peered":"2024-09-10T22:57:33.495716+0000","last_clean":"2024-09-10T22:57:33.495716+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:33.495716+0000","last_undegraded":"2024-09-10T22:57:33.495716+0000","last_fullsized":"2024-09-10T22:57:33.495716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484068+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:57:33.484068+0000","last_peered":"2024-09-10T22:57:33.484068+0000","last_clean":"2024-09-10T22:57:33.484068+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:57:33.484068+0000","last_undegraded":"2024-09-10T22:57:33.484068+0000","last_fullsized":"2024-09-10T22:57:33.484068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490385+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:57:33.490385+0000","last_peered":"2024-09-10T22:57:33.490385+0000","last_clean":"2024-09-10T22:57:33.490385+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:57:33.490385+0000","last_undegraded":"2024-09-10T22:57:33.490385+0000","last_fullsized":"2024-09-10T22:57:33.490385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.968470+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:57:33.968470+0000","last_peered":"2024-09-10T22:57:33.968470+0000","last_clean":"2024-09-10T22:57:33.968470+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:57:33.968470+0000","last_undegraded":"2024-09-10T22:57:33.968470+0000","last_fullsized":"2024-09-10T22:57:33.968470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636127+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:57:33.636127+0000","last_peered":"2024-09-10T22:57:33.636127+0000","last_clean":"2024-09-10T22:57:33.636127+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:57:33.636127+0000","last_undegraded":"2024-09-10T22:57:33.636127+0000","last_fullsized":"2024-09-10T22:57:33.636127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491949+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:57:33.491949+0000","last_peered":"2024-09-10T22:57:33.491949+0000","last_clean":"2024-09-10T22:57:33.491949+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:57:33.491949+0000","last_undegraded":"2024-09-10T22:57:33.491949+0000","last_fullsized":"2024-09-10T22:57:33.491949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:27.586585+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:58:27.586585+0000","last_peered":"2024-09-10T22:58:27.586585+0000","last_clean":"2024-09-10T22:58:27.586585+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:58:27.586585+0000","last_undegraded":"2024-09-10T22:58:27.586585+0000","last_fullsized":"2024-09-10T22:58:27.586585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485425+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:57:33.485425+0000","last_peered":"2024-09-10T22:57:33.485425+0000","last_clean":"2024-09-10T22:57:33.485425+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:57:33.485425+0000","last_undegraded":"2024-09-10T22:57:33.485425+0000","last_fullsized":"2024-09-10T22:57:33.485425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487381+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:57:33.487381+0000","last_peered":"2024-09-10T22:57:33.487381+0000","last_clean":"2024-09-10T22:57:33.487381+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:57:33.487381+0000","last_undegraded":"2024-09-10T22:57:33.487381+0000","last_fullsized":"2024-09-10T22:57:33.487381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484551+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:57:33.484551+0000","last_peered":"2024-09-10T22:57:33.484551+0000","last_clean":"2024-09-10T22:57:33.484551+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:57:33.484551+0000","last_undegraded":"2024-09-10T22:57:33.484551+0000","last_fullsized":"2024-09-10T22:57:33.484551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488302+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:33.488302+0000","last_peered":"2024-09-10T22:57:33.488302+0000","last_clean":"2024-09-10T22:57:33.488302+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:33.488302+0000","last_undegraded":"2024-09-10T22:57:33.488302+0000","last_fullsized":"2024-09-10T22:57:33.488302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635806+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:57:33.635806+0000","last_peered":"2024-09-10T22:57:33.635806+0000","last_clean":"2024-09-10T22:57:33.635806+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:57:33.635806+0000","last_undegraded":"2024-09-10T22:57:33.635806+0000","last_fullsized":"2024-09-10T22:57:33.635806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489715+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:57:33.489715+0000","last_peered":"2024-09-10T22:57:33.489715+0000","last_clean":"2024-09-10T22:57:33.489715+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:57:33.489715+0000","last_undegraded":"2024-09-10T22:57:33.489715+0000","last_fullsized":"2024-09-10T22:57:33.489715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485286+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:57:33.485286+0000","last_peered":"2024-09-10T22:57:33.485286+0000","last_clean":"2024-09-10T22:57:33.485286+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:57:33.485286+0000","last_undegraded":"2024-09-10T22:57:33.485286+0000","last_fullsized":"2024-09-10T22:57:33.485286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495545+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:57:33.495545+0000","last_peered":"2024-09-10T22:57:33.495545+0000","last_clean":"2024-09-10T22:57:33.495545+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:57:33.495545+0000","last_undegraded":"2024-09-10T22:57:33.495545+0000","last_fullsized":"2024-09-10T22:57:33.495545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":233,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:30.592138+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:58:30.592138+0000","last_peered":"2024-09-10T22:58:30.592138+0000","last_clean":"2024-09-10T22:58:30.592138+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:58:30.592138+0000","last_undegraded":"2024-09-10T22:58:30.592138+0000","last_fullsized":"2024-09-10T22:58:30.592138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485135+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:57:33.485135+0000","last_peered":"2024-09-10T22:57:33.485135+0000","last_clean":"2024-09-10T22:57:33.485135+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:57:33.485135+0000","last_undegraded":"2024-09-10T22:57:33.485135+0000","last_fullsized":"2024-09-10T22:57:33.485135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484785+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:57:33.484785+0000","last_peered":"2024-09-10T22:57:33.484785+0000","last_clean":"2024-09-10T22:57:33.484785+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:57:33.484785+0000","last_undegraded":"2024-09-10T22:57:33.484785+0000","last_fullsized":"2024-09-10T22:57:33.484785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485645+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:57:33.485645+0000","last_peered":"2024-09-10T22:57:33.485645+0000","last_clean":"2024-09-10T22:57:33.485645+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:57:33.485645+0000","last_undegraded":"2024-09-10T22:57:33.485645+0000","last_fullsized":"2024-09-10T22:57:33.485645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933615+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:57:33.933615+0000","last_peered":"2024-09-10T22:57:33.933615+0000","last_clean":"2024-09-10T22:57:33.933615+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:57:33.933615+0000","last_undegraded":"2024-09-10T22:57:33.933615+0000","last_fullsized":"2024-09-10T22:57:33.933615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492566+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:57:33.492566+0000","last_peered":"2024-09-10T22:57:33.492566+0000","last_clean":"2024-09-10T22:57:33.492566+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:57:33.492566+0000","last_undegraded":"2024-09-10T22:57:33.492566+0000","last_fullsized":"2024-09-10T22:57:33.492566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490487+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:57:33.490487+0000","last_peered":"2024-09-10T22:57:33.490487+0000","last_clean":"2024-09-10T22:57:33.490487+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:57:33.490487+0000","last_undegraded":"2024-09-10T22:57:33.490487+0000","last_fullsized":"2024-09-10T22:57:33.490487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484599+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:57:33.484599+0000","last_peered":"2024-09-10T22:57:33.484599+0000","last_clean":"2024-09-10T22:57:33.484599+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:57:33.484599+0000","last_undegraded":"2024-09-10T22:57:33.484599+0000","last_fullsized":"2024-09-10T22:57:33.484599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485175+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:57:33.485175+0000","last_peered":"2024-09-10T22:57:33.485175+0000","last_clean":"2024-09-10T22:57:33.485175+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:57:33.485175+0000","last_undegraded":"2024-09-10T22:57:33.485175+0000","last_fullsized":"2024-09-10T22:57:33.485175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:29.590216+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:58:29.590216+0000","last_peered":"2024-09-10T22:58:29.590216+0000","last_clean":"2024-09-10T22:58:29.590216+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:58:29.590216+0000","last_undegraded":"2024-09-10T22:58:29.590216+0000","last_fullsized":"2024-09-10T22:58:29.590216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482848+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:57:33.482848+0000","last_peered":"2024-09-10T22:57:33.482848+0000","last_clean":"2024-09-10T22:57:33.482848+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:57:33.482848+0000","last_undegraded":"2024-09-10T22:57:33.482848+0000","last_fullsized":"2024-09-10T22:57:33.482848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492234+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:57:33.492234+0000","last_peered":"2024-09-10T22:57:33.492234+0000","last_clean":"2024-09-10T22:57:33.492234+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:57:33.492234+0000","last_undegraded":"2024-09-10T22:57:33.492234+0000","last_fullsized":"2024-09-10T22:57:33.492234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484686+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:57:33.484686+0000","last_peered":"2024-09-10T22:57:33.484686+0000","last_clean":"2024-09-10T22:57:33.484686+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:57:33.484686+0000","last_undegraded":"2024-09-10T22:57:33.484686+0000","last_fullsized":"2024-09-10T22:57:33.484686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484688+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:57:33.484688+0000","last_peered":"2024-09-10T22:57:33.484688+0000","last_clean":"2024-09-10T22:57:33.484688+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:57:33.484688+0000","last_undegraded":"2024-09-10T22:57:33.484688+0000","last_fullsized":"2024-09-10T22:57:33.484688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488441+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:33.488441+0000","last_peered":"2024-09-10T22:57:33.488441+0000","last_clean":"2024-09-10T22:57:33.488441+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:33.488441+0000","last_undegraded":"2024-09-10T22:57:33.488441+0000","last_fullsized":"2024-09-10T22:57:33.488441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635694+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:33.635694+0000","last_peered":"2024-09-10T22:57:33.635694+0000","last_clean":"2024-09-10T22:57:33.635694+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:33.635694+0000","last_undegraded":"2024-09-10T22:57:33.635694+0000","last_fullsized":"2024-09-10T22:57:33.635694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489621+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:33.489621+0000","last_peered":"2024-09-10T22:57:33.489621+0000","last_clean":"2024-09-10T22:57:33.489621+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:33.489621+0000","last_undegraded":"2024-09-10T22:57:33.489621+0000","last_fullsized":"2024-09-10T22:57:33.489621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484446+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:57:33.484446+0000","last_peered":"2024-09-10T22:57:33.484446+0000","last_clean":"2024-09-10T22:57:33.484446+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:57:33.484446+0000","last_undegraded":"2024-09-10T22:57:33.484446+0000","last_fullsized":"2024-09-10T22:57:33.484446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489337+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:57:33.489337+0000","last_peered":"2024-09-10T22:57:33.489337+0000","last_clean":"2024-09-10T22:57:33.489337+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:57:33.489337+0000","last_undegraded":"2024-09-10T22:57:33.489337+0000","last_fullsized":"2024-09-10T22:57:33.489337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:27.588023+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:58:27.588023+0000","last_peered":"2024-09-10T22:58:27.588023+0000","last_clean":"2024-09-10T22:58:27.588023+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:58:27.588023+0000","last_undegraded":"2024-09-10T22:58:27.588023+0000","last_fullsized":"2024-09-10T22:58:27.588023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494636+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:57:33.494636+0000","last_peered":"2024-09-10T22:57:33.494636+0000","last_clean":"2024-09-10T22:57:33.494636+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:57:33.494636+0000","last_undegraded":"2024-09-10T22:57:33.494636+0000","last_fullsized":"2024-09-10T22:57:33.494636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490354+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:57:33.490354+0000","last_peered":"2024-09-10T22:57:33.490354+0000","last_clean":"2024-09-10T22:57:33.490354+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:57:33.490354+0000","last_undegraded":"2024-09-10T22:57:33.490354+0000","last_fullsized":"2024-09-10T22:57:33.490354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489532+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:57:33.489532+0000","last_peered":"2024-09-10T22:57:33.489532+0000","last_clean":"2024-09-10T22:57:33.489532+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:57:33.489532+0000","last_undegraded":"2024-09-10T22:57:33.489532+0000","last_fullsized":"2024-09-10T22:57:33.489532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488650+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:57:33.488650+0000","last_peered":"2024-09-10T22:57:33.488650+0000","last_clean":"2024-09-10T22:57:33.488650+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:57:33.488650+0000","last_undegraded":"2024-09-10T22:57:33.488650+0000","last_fullsized":"2024-09-10T22:57:33.488650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488541+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:57:33.488541+0000","last_peered":"2024-09-10T22:57:33.488541+0000","last_clean":"2024-09-10T22:57:33.488541+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:57:33.488541+0000","last_undegraded":"2024-09-10T22:57:33.488541+0000","last_fullsized":"2024-09-10T22:57:33.488541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492474+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:57:33.492474+0000","last_peered":"2024-09-10T22:57:33.492474+0000","last_clean":"2024-09-10T22:57:33.492474+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:57:33.492474+0000","last_undegraded":"2024-09-10T22:57:33.492474+0000","last_fullsized":"2024-09-10T22:57:33.492474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636120+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:57:33.636120+0000","last_peered":"2024-09-10T22:57:33.636120+0000","last_clean":"2024-09-10T22:57:33.636120+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:57:33.636120+0000","last_undegraded":"2024-09-10T22:57:33.636120+0000","last_fullsized":"2024-09-10T22:57:33.636120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489380+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:57:33.489380+0000","last_peered":"2024-09-10T22:57:33.489380+0000","last_clean":"2024-09-10T22:57:33.489380+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:57:33.489380+0000","last_undegraded":"2024-09-10T22:57:33.489380+0000","last_fullsized":"2024-09-10T22:57:33.489380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489302+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:57:33.489302+0000","last_peered":"2024-09-10T22:57:33.489302+0000","last_clean":"2024-09-10T22:57:33.489302+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:57:33.489302+0000","last_undegraded":"2024-09-10T22:57:33.489302+0000","last_fullsized":"2024-09-10T22:57:33.489302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.483443+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:57:33.483443+0000","last_peered":"2024-09-10T22:57:33.483443+0000","last_clean":"2024-09-10T22:57:33.483443+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:57:33.483443+0000","last_undegraded":"2024-09-10T22:57:33.483443+0000","last_fullsized":"2024-09-10T22:57:33.483443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484150+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:57:33.484150+0000","last_peered":"2024-09-10T22:57:33.484150+0000","last_clean":"2024-09-10T22:57:33.484150+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:57:33.484150+0000","last_undegraded":"2024-09-10T22:57:33.484150+0000","last_fullsized":"2024-09-10T22:57:33.484150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482935+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:57:33.482935+0000","last_peered":"2024-09-10T22:57:33.482935+0000","last_clean":"2024-09-10T22:57:33.482935+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:57:33.482935+0000","last_undegraded":"2024-09-10T22:57:33.482935+0000","last_fullsized":"2024-09-10T22:57:33.482935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484667+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:57:33.484667+0000","last_peered":"2024-09-10T22:57:33.484667+0000","last_clean":"2024-09-10T22:57:33.484667+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:57:33.484667+0000","last_undegraded":"2024-09-10T22:57:33.484667+0000","last_fullsized":"2024-09-10T22:57:33.484667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635935+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:57:33.635935+0000","last_peered":"2024-09-10T22:57:33.635935+0000","last_clean":"2024-09-10T22:57:33.635935+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:57:33.635935+0000","last_undegraded":"2024-09-10T22:57:33.635935+0000","last_fullsized":"2024-09-10T22:57:33.635935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488423+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:57:33.488423+0000","last_peered":"2024-09-10T22:57:33.488423+0000","last_clean":"2024-09-10T22:57:33.488423+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:57:33.488423+0000","last_undegraded":"2024-09-10T22:57:33.488423+0000","last_fullsized":"2024-09-10T22:57:33.488423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490516+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:57:33.490516+0000","last_peered":"2024-09-10T22:57:33.490516+0000","last_clean":"2024-09-10T22:57:33.490516+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:57:33.490516+0000","last_undegraded":"2024-09-10T22:57:33.490516+0000","last_fullsized":"2024-09-10T22:57:33.490516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485363+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:57:33.485363+0000","last_peered":"2024-09-10T22:57:33.485363+0000","last_clean":"2024-09-10T22:57:33.485363+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:57:33.485363+0000","last_undegraded":"2024-09-10T22:57:33.485363+0000","last_fullsized":"2024-09-10T22:57:33.485363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489898+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:57:33.489898+0000","last_peered":"2024-09-10T22:57:33.489898+0000","last_clean":"2024-09-10T22:57:33.489898+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:57:33.489898+0000","last_undegraded":"2024-09-10T22:57:33.489898+0000","last_fullsized":"2024-09-10T22:57:33.489898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484224+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:57:33.484224+0000","last_peered":"2024-09-10T22:57:33.484224+0000","last_clean":"2024-09-10T22:57:33.484224+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:57:33.484224+0000","last_undegraded":"2024-09-10T22:57:33.484224+0000","last_fullsized":"2024-09-10T22:57:33.484224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491907+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:57:33.491907+0000","last_peered":"2024-09-10T22:57:33.491907+0000","last_clean":"2024-09-10T22:57:33.491907+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:57:33.491907+0000","last_undegraded":"2024-09-10T22:57:33.491907+0000","last_fullsized":"2024-09-10T22:57:33.491907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491892+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:57:33.491892+0000","last_peered":"2024-09-10T22:57:33.491892+0000","last_clean":"2024-09-10T22:57:33.491892+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:57:33.491892+0000","last_undegraded":"2024-09-10T22:57:33.491892+0000","last_fullsized":"2024-09-10T22:57:33.491892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635943+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:57:33.635943+0000","last_peered":"2024-09-10T22:57:33.635943+0000","last_clean":"2024-09-10T22:57:33.635943+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:57:33.635943+0000","last_undegraded":"2024-09-10T22:57:33.635943+0000","last_fullsized":"2024-09-10T22:57:33.635943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934287+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:57:33.934287+0000","last_peered":"2024-09-10T22:57:33.934287+0000","last_clean":"2024-09-10T22:57:33.934287+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:57:33.934287+0000","last_undegraded":"2024-09-10T22:57:33.934287+0000","last_fullsized":"2024-09-10T22:57:33.934287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487645+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:57:33.487645+0000","last_peered":"2024-09-10T22:57:33.487645+0000","last_clean":"2024-09-10T22:57:33.487645+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:57:33.487645+0000","last_undegraded":"2024-09-10T22:57:33.487645+0000","last_fullsized":"2024-09-10T22:57:33.487645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490348+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:57:33.490348+0000","last_peered":"2024-09-10T22:57:33.490348+0000","last_clean":"2024-09-10T22:57:33.490348+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:57:33.490348+0000","last_undegraded":"2024-09-10T22:57:33.490348+0000","last_fullsized":"2024-09-10T22:57:33.490348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485308+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:57:33.485308+0000","last_peered":"2024-09-10T22:57:33.485308+0000","last_clean":"2024-09-10T22:57:33.485308+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:57:33.485308+0000","last_undegraded":"2024-09-10T22:57:33.485308+0000","last_fullsized":"2024-09-10T22:57:33.485308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.482841+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:57:33.482841+0000","last_peered":"2024-09-10T22:57:33.482841+0000","last_clean":"2024-09-10T22:57:33.482841+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:57:33.482841+0000","last_undegraded":"2024-09-10T22:57:33.482841+0000","last_fullsized":"2024-09-10T22:57:33.482841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487369+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:57:33.487369+0000","last_peered":"2024-09-10T22:57:33.487369+0000","last_clean":"2024-09-10T22:57:33.487369+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:57:33.487369+0000","last_undegraded":"2024-09-10T22:57:33.487369+0000","last_fullsized":"2024-09-10T22:57:33.487369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495990+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:57:33.495990+0000","last_peered":"2024-09-10T22:57:33.495990+0000","last_clean":"2024-09-10T22:57:33.495990+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:57:33.495990+0000","last_undegraded":"2024-09-10T22:57:33.495990+0000","last_fullsized":"2024-09-10T22:57:33.495990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494336+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:57:33.494336+0000","last_peered":"2024-09-10T22:57:33.494336+0000","last_clean":"2024-09-10T22:57:33.494336+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:57:33.494336+0000","last_undegraded":"2024-09-10T22:57:33.494336+0000","last_fullsized":"2024-09-10T22:57:33.494336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933913+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:57:33.933913+0000","last_peered":"2024-09-10T22:57:33.933913+0000","last_clean":"2024-09-10T22:57:33.933913+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:57:33.933913+0000","last_undegraded":"2024-09-10T22:57:33.933913+0000","last_fullsized":"2024-09-10T22:57:33.933913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492318+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:57:33.492318+0000","last_peered":"2024-09-10T22:57:33.492318+0000","last_clean":"2024-09-10T22:57:33.492318+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:57:33.492318+0000","last_undegraded":"2024-09-10T22:57:33.492318+0000","last_fullsized":"2024-09-10T22:57:33.492318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492424+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:57:33.492424+0000","last_peered":"2024-09-10T22:57:33.492424+0000","last_clean":"2024-09-10T22:57:33.492424+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:57:33.492424+0000","last_undegraded":"2024-09-10T22:57:33.492424+0000","last_fullsized":"2024-09-10T22:57:33.492424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489826+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:57:33.489826+0000","last_peered":"2024-09-10T22:57:33.489826+0000","last_clean":"2024-09-10T22:57:33.489826+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:57:33.489826+0000","last_undegraded":"2024-09-10T22:57:33.489826+0000","last_fullsized":"2024-09-10T22:57:33.489826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":490,"num_read_kb":318,"num_write":168,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":499051,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1148279,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":503271,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":498983,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":1095004,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561096,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":494351,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":1090592,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":499113,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117081,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":494545,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378721,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":556949,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":557346,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":495739,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:58:33.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:58:34.048 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:34.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:58:34.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T22:58:34.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:58:34.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-10T22:58:34.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T22:58:34.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:58:34.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:34 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:58:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T22:58:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:58:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-10T22:58:34.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T22:58:34.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:58:34.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:34 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:34 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:34.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 -- 172.21.15.67:0/2257025821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 msgr2=0x7fca6c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:34.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 --2- 172.21.15.67:0/2257025821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c10f950 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ca0 tx=0x7fca6402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 -- 172.21.15.67:0/2257025821 shutdown_connections 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 --2- 172.21.15.67:0/2257025821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c10f950 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 --2- 172.21.15.67:0/2257025821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c1087d0 0x7fca6c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 --2- 172.21.15.67:0/2257025821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c102dc0 0x7fca6c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 -- 172.21.15.67:0/2257025821 >> 172.21.15.67:0/2257025821 conn(0x7fca6c0fe830 msgr2=0x7fca6c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 -- 172.21.15.67:0/2257025821 shutdown_connections 2024-09-10T22:58:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.630+0000 7fca72200640 1 -- 172.21.15.67:0/2257025821 wait complete. 2024-09-10T22:58:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.631+0000 7fca72200640 1 Processor -- start 2024-09-10T22:58:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.631+0000 7fca72200640 1 -- start start 2024-09-10T22:58:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c102dc0 0x7fca6c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 0x7fca6c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112cd0 con 0x7fca6c109150 2024-09-10T22:58:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112b50 con 0x7fca6c1087d0 2024-09-10T22:58:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca72200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c112e50 con 0x7fca6c102dc0 2024-09-10T22:58:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca6b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c102dc0 0x7fca6c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60382/0 (socket says 172.21.15.67:60382) 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca6be00640 1 -- 172.21.15.67:0/2888654483 learned_addr learned my addr 172.21.15.67:0/2888654483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.632+0000 7fca6aa00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 0x7fca6c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 -- 172.21.15.67:0/2888654483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c102dc0 msgr2=0x7fca6c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c102dc0 0x7fca6c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 -- 172.21.15.67:0/2888654483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 msgr2=0x7fca6c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 0x7fca6c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 -- 172.21.15.67:0/2888654483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c1a7580 con 0x7fca6c109150 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6aa00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 0x7fca6c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.633+0000 7fca6be00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca64005e30 tx=0x7fca640093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.634+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64032d10 con 0x7fca6c109150 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.634+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca64032eb0 con 0x7fca6c109150 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.634+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c1a7810 con 0x7fca6c109150 2024-09-10T22:58:34.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.634+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64030b20 con 0x7fca6c109150 2024-09-10T22:58:34.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.634+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c1a7da0 con 0x7fca6c109150 2024-09-10T22:58:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.635+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c1031a0 con 0x7fca6c109150 2024-09-10T22:58:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.637+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fca64030cc0 con 0x7fca6c109150 2024-09-10T22:58:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.639+0000 7fca63e00640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 0x7fca500b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.639+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fca64107360 con 0x7fca6c109150 2024-09-10T22:58:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.639+0000 7fca6b400640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 0x7fca500b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.640+0000 7fca6b400640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 0x7fca500b7900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009bd0 tx=0x7fca5c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.643+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fca640cdbb0 con 0x7fca6c109150 2024-09-10T22:58:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.844+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca6c063590 con 0x7fca500b5440 2024-09-10T22:58:34.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.850+0000 7fca63e00640 1 -- 172.21.15.67:0/2888654483 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340783 (secure 0 0 0) 0x7fca6c063590 con 0x7fca500b5440 2024-09-10T22:58:34.857 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 msgr2=0x7fca500b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 0x7fca500b7900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009bd0 tx=0x7fca5c005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 msgr2=0x7fca6c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca64005e30 tx=0x7fca640093d0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 shutdown_connections 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fca500b5440 0x7fca500b7900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca6c109150 0x7fca6c075f20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca6c1087d0 0x7fca6c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 --2- 172.21.15.67:0/2888654483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca6c102dc0 0x7fca6c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 >> 172.21.15.67:0/2888654483 conn(0x7fca6c0fe830 msgr2=0x7fca6c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 shutdown_connections 2024-09-10T22:58:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:34.855+0000 7fca72200640 1 -- 172.21.15.67:0/2888654483 wait complete. 2024-09-10T22:58:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T22:58:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:58:35.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:35.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:35 smithi092 ceph-mon[34223]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:35.532 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:35 smithi145 ceph-mon[34397]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:58:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:35 smithi067 ceph-mon[30939]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:58:35.727 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-10T22:58:34.182884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656816,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":577,"num_read_kb":388,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":540,"ondisk_log_size":540,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356836,"kb_used_data":11704,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584988,"statfs":{"total":1151940427776,"available":1151575027712,"internally_reserved":0,"allocated":11984896,"data_stored":7953696,"data_compressed":397337,"data_compressed_allocated":536576,"data_compressed_original":2572288,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":16,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.092344"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485634+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:57:33.485634+0000","last_peered":"2024-09-10T22:57:33.485634+0000","last_clean":"2024-09-10T22:57:33.485634+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:57:33.485634+0000","last_undegraded":"2024-09-10T22:57:33.485634+0000","last_fullsized":"2024-09-10T22:57:33.485634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185909+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:58:33.185909+0000","last_peered":"2024-09-10T22:58:33.185909+0000","last_clean":"2024-09-10T22:58:33.185909+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:58:33.185909+0000","last_undegraded":"2024-09-10T22:58:33.185909+0000","last_fullsized":"2024-09-10T22:58:33.185909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182392+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:58:33.182392+0000","last_peered":"2024-09-10T22:58:33.182392+0000","last_clean":"2024-09-10T22:58:33.182392+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:58:33.182392+0000","last_undegraded":"2024-09-10T22:58:33.182392+0000","last_fullsized":"2024-09-10T22:58:33.182392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.186222+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:58:33.186222+0000","last_peered":"2024-09-10T22:58:33.186222+0000","last_clean":"2024-09-10T22:58:33.186222+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:58:33.186222+0000","last_undegraded":"2024-09-10T22:58:33.186222+0000","last_fullsized":"2024-09-10T22:58:33.186222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492613+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:57:33.492613+0000","last_peered":"2024-09-10T22:57:33.492613+0000","last_clean":"2024-09-10T22:57:33.492613+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:57:33.492613+0000","last_undegraded":"2024-09-10T22:57:33.492613+0000","last_fullsized":"2024-09-10T22:57:33.492613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933763+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:57:33.933763+0000","last_peered":"2024-09-10T22:57:33.933763+0000","last_clean":"2024-09-10T22:57:33.933763+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:57:33.933763+0000","last_undegraded":"2024-09-10T22:57:33.933763+0000","last_fullsized":"2024-09-10T22:57:33.933763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495695+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:57:33.495695+0000","last_peered":"2024-09-10T22:57:33.495695+0000","last_clean":"2024-09-10T22:57:33.495695+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:57:33.495695+0000","last_undegraded":"2024-09-10T22:57:33.495695+0000","last_fullsized":"2024-09-10T22:57:33.495695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183412+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:58:33.183412+0000","last_peered":"2024-09-10T22:58:33.183412+0000","last_clean":"2024-09-10T22:58:33.183412+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:58:33.183412+0000","last_undegraded":"2024-09-10T22:58:33.183412+0000","last_fullsized":"2024-09-10T22:58:33.183412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933235+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:57:33.933235+0000","last_peered":"2024-09-10T22:57:33.933235+0000","last_clean":"2024-09-10T22:57:33.933235+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:57:33.933235+0000","last_undegraded":"2024-09-10T22:57:33.933235+0000","last_fullsized":"2024-09-10T22:57:33.933235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.499755+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:57:33.499755+0000","last_peered":"2024-09-10T22:57:33.499755+0000","last_clean":"2024-09-10T22:57:33.499755+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:57:33.499755+0000","last_undegraded":"2024-09-10T22:57:33.499755+0000","last_fullsized":"2024-09-10T22:57:33.499755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183915+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:58:33.183915+0000","last_peered":"2024-09-10T22:58:33.183915+0000","last_clean":"2024-09-10T22:58:33.183915+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:58:33.183915+0000","last_undegraded":"2024-09-10T22:58:33.183915+0000","last_fullsized":"2024-09-10T22:58:33.183915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933169+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:57:33.933169+0000","last_peered":"2024-09-10T22:57:33.933169+0000","last_clean":"2024-09-10T22:57:33.933169+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:57:33.933169+0000","last_undegraded":"2024-09-10T22:57:33.933169+0000","last_fullsized":"2024-09-10T22:57:33.933169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232800+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:58:33.232800+0000","last_peered":"2024-09-10T22:58:33.232800+0000","last_clean":"2024-09-10T22:58:33.232800+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:58:33.232800+0000","last_undegraded":"2024-09-10T22:58:33.232800+0000","last_fullsized":"2024-09-10T22:58:33.232800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494981+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:57:33.494981+0000","last_peered":"2024-09-10T22:57:33.494981+0000","last_clean":"2024-09-10T22:57:33.494981+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:57:33.494981+0000","last_undegraded":"2024-09-10T22:57:33.494981+0000","last_fullsized":"2024-09-10T22:57:33.494981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933680+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:57:33.933680+0000","last_peered":"2024-09-10T22:57:33.933680+0000","last_clean":"2024-09-10T22:57:33.933680+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:57:33.933680+0000","last_undegraded":"2024-09-10T22:57:33.933680+0000","last_fullsized":"2024-09-10T22:57:33.933680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185780+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:58:33.185780+0000","last_peered":"2024-09-10T22:58:33.185780+0000","last_clean":"2024-09-10T22:58:33.185780+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:58:33.185780+0000","last_undegraded":"2024-09-10T22:58:33.185780+0000","last_fullsized":"2024-09-10T22:58:33.185780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182566+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:58:33.182566+0000","last_peered":"2024-09-10T22:58:33.182566+0000","last_clean":"2024-09-10T22:58:33.182566+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:58:33.182566+0000","last_undegraded":"2024-09-10T22:58:33.182566+0000","last_fullsized":"2024-09-10T22:58:33.182566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933688+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:57:33.933688+0000","last_peered":"2024-09-10T22:57:33.933688+0000","last_clean":"2024-09-10T22:57:33.933688+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:57:33.933688+0000","last_undegraded":"2024-09-10T22:57:33.933688+0000","last_fullsized":"2024-09-10T22:57:33.933688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.495929+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:58:32.495929+0000","last_peered":"2024-09-10T22:58:32.495929+0000","last_clean":"2024-09-10T22:58:32.495929+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:58:32.495929+0000","last_undegraded":"2024-09-10T22:58:32.495929+0000","last_fullsized":"2024-09-10T22:58:32.495929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185777+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:58:33.185777+0000","last_peered":"2024-09-10T22:58:33.185777+0000","last_clean":"2024-09-10T22:58:33.185777+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:58:33.185777+0000","last_undegraded":"2024-09-10T22:58:33.185777+0000","last_fullsized":"2024-09-10T22:58:33.185777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485365+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:57:33.485365+0000","last_peered":"2024-09-10T22:57:33.485365+0000","last_clean":"2024-09-10T22:57:33.485365+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:57:33.485365+0000","last_undegraded":"2024-09-10T22:57:33.485365+0000","last_fullsized":"2024-09-10T22:57:33.485365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494037+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:57:33.494037+0000","last_peered":"2024-09-10T22:57:33.494037+0000","last_clean":"2024-09-10T22:57:33.494037+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:57:33.494037+0000","last_undegraded":"2024-09-10T22:57:33.494037+0000","last_fullsized":"2024-09-10T22:57:33.494037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182543+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:58:33.182543+0000","last_peered":"2024-09-10T22:58:33.182543+0000","last_clean":"2024-09-10T22:58:33.182543+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:58:33.182543+0000","last_undegraded":"2024-09-10T22:58:33.182543+0000","last_fullsized":"2024-09-10T22:58:33.182543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933699+0000","last_change":"2024-09-10T22:56:27.965396+0000","last_active":"2024-09-10T22:57:33.933699+0000","last_peered":"2024-09-10T22:57:33.933699+0000","last_clean":"2024-09-10T22:57:33.933699+0000","last_became_active":"2024-09-10T22:56:27.964744+0000","last_became_peered":"2024-09-10T22:56:27.964744+0000","last_unstale":"2024-09-10T22:57:33.933699+0000","last_undegraded":"2024-09-10T22:57:33.933699+0000","last_fullsized":"2024-09-10T22:57:33.933699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:26.622928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492482+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:57:33.492482+0000","last_peered":"2024-09-10T22:57:33.492482+0000","last_clean":"2024-09-10T22:57:33.492482+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:57:33.492482+0000","last_undegraded":"2024-09-10T22:57:33.492482+0000","last_fullsized":"2024-09-10T22:57:33.492482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934332+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:57:33.934332+0000","last_peered":"2024-09-10T22:57:33.934332+0000","last_clean":"2024-09-10T22:57:33.934332+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:57:33.934332+0000","last_undegraded":"2024-09-10T22:57:33.934332+0000","last_fullsized":"2024-09-10T22:57:33.934332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182641+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:58:33.182641+0000","last_peered":"2024-09-10T22:58:33.182641+0000","last_clean":"2024-09-10T22:58:33.182641+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:58:33.182641+0000","last_undegraded":"2024-09-10T22:58:33.182641+0000","last_fullsized":"2024-09-10T22:58:33.182641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.186067+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:58:33.186067+0000","last_peered":"2024-09-10T22:58:33.186067+0000","last_clean":"2024-09-10T22:58:33.186067+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:58:33.186067+0000","last_undegraded":"2024-09-10T22:58:33.186067+0000","last_fullsized":"2024-09-10T22:58:33.186067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485444+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:57:33.485444+0000","last_peered":"2024-09-10T22:57:33.485444+0000","last_clean":"2024-09-10T22:57:33.485444+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:57:33.485444+0000","last_undegraded":"2024-09-10T22:57:33.485444+0000","last_fullsized":"2024-09-10T22:57:33.485444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185158+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:58:33.185158+0000","last_peered":"2024-09-10T22:58:33.185158+0000","last_clean":"2024-09-10T22:58:33.185158+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:58:33.185158+0000","last_undegraded":"2024-09-10T22:58:33.185158+0000","last_fullsized":"2024-09-10T22:58:33.185158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636096+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:57:33.636096+0000","last_peered":"2024-09-10T22:57:33.636096+0000","last_clean":"2024-09-10T22:57:33.636096+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:57:33.636096+0000","last_undegraded":"2024-09-10T22:57:33.636096+0000","last_fullsized":"2024-09-10T22:57:33.636096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495769+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:57:33.495769+0000","last_peered":"2024-09-10T22:57:33.495769+0000","last_clean":"2024-09-10T22:57:33.495769+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:57:33.495769+0000","last_undegraded":"2024-09-10T22:57:33.495769+0000","last_fullsized":"2024-09-10T22:57:33.495769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.487790+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:57:33.487790+0000","last_peered":"2024-09-10T22:57:33.487790+0000","last_clean":"2024-09-10T22:57:33.487790+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:57:33.487790+0000","last_undegraded":"2024-09-10T22:57:33.487790+0000","last_fullsized":"2024-09-10T22:57:33.487790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489169+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:57:33.489169+0000","last_peered":"2024-09-10T22:57:33.489169+0000","last_clean":"2024-09-10T22:57:33.489169+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:57:33.489169+0000","last_undegraded":"2024-09-10T22:57:33.489169+0000","last_fullsized":"2024-09-10T22:57:33.489169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485206+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:57:33.485206+0000","last_peered":"2024-09-10T22:57:33.485206+0000","last_clean":"2024-09-10T22:57:33.485206+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:57:33.485206+0000","last_undegraded":"2024-09-10T22:57:33.485206+0000","last_fullsized":"2024-09-10T22:57:33.485206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.493448+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:57:33.493448+0000","last_peered":"2024-09-10T22:57:33.493448+0000","last_clean":"2024-09-10T22:57:33.493448+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:57:33.493448+0000","last_undegraded":"2024-09-10T22:57:33.493448+0000","last_fullsized":"2024-09-10T22:57:33.493448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490145+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:57:33.490145+0000","last_peered":"2024-09-10T22:57:33.490145+0000","last_clean":"2024-09-10T22:57:33.490145+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:57:33.490145+0000","last_undegraded":"2024-09-10T22:57:33.490145+0000","last_fullsized":"2024-09-10T22:57:33.490145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232956+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:58:33.232956+0000","last_peered":"2024-09-10T22:58:33.232956+0000","last_clean":"2024-09-10T22:58:33.232956+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:58:33.232956+0000","last_undegraded":"2024-09-10T22:58:33.232956+0000","last_fullsized":"2024-09-10T22:58:33.232956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635906+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:57:33.635906+0000","last_peered":"2024-09-10T22:57:33.635906+0000","last_clean":"2024-09-10T22:57:33.635906+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:57:33.635906+0000","last_undegraded":"2024-09-10T22:57:33.635906+0000","last_fullsized":"2024-09-10T22:57:33.635906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182086+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:58:33.182086+0000","last_peered":"2024-09-10T22:58:33.182086+0000","last_clean":"2024-09-10T22:58:33.182086+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:58:33.182086+0000","last_undegraded":"2024-09-10T22:58:33.182086+0000","last_fullsized":"2024-09-10T22:58:33.182086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:28.587447+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:58:28.587447+0000","last_peered":"2024-09-10T22:58:28.587447+0000","last_clean":"2024-09-10T22:58:28.587447+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:58:28.587447+0000","last_undegraded":"2024-09-10T22:58:28.587447+0000","last_fullsized":"2024-09-10T22:58:28.587447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636145+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:57:33.636145+0000","last_peered":"2024-09-10T22:57:33.636145+0000","last_clean":"2024-09-10T22:57:33.636145+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:57:33.636145+0000","last_undegraded":"2024-09-10T22:57:33.636145+0000","last_fullsized":"2024-09-10T22:57:33.636145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635709+0000","last_change":"2024-09-10T22:56:18.923575+0000","last_active":"2024-09-10T22:57:33.635709+0000","last_peered":"2024-09-10T22:57:33.635709+0000","last_clean":"2024-09-10T22:57:33.635709+0000","last_became_active":"2024-09-10T22:56:18.921430+0000","last_became_peered":"2024-09-10T22:56:18.921430+0000","last_unstale":"2024-09-10T22:57:33.635709+0000","last_undegraded":"2024-09-10T22:57:33.635709+0000","last_fullsized":"2024-09-10T22:57:33.635709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:43.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232590+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:58:33.232590+0000","last_peered":"2024-09-10T22:58:33.232590+0000","last_clean":"2024-09-10T22:58:33.232590+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:58:33.232590+0000","last_undegraded":"2024-09-10T22:58:33.232590+0000","last_fullsized":"2024-09-10T22:58:33.232590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184613+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:58:33.184613+0000","last_peered":"2024-09-10T22:58:33.184613+0000","last_clean":"2024-09-10T22:58:33.184613+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:58:33.184613+0000","last_undegraded":"2024-09-10T22:58:33.184613+0000","last_fullsized":"2024-09-10T22:58:33.184613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489346+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:57:33.489346+0000","last_peered":"2024-09-10T22:57:33.489346+0000","last_clean":"2024-09-10T22:57:33.489346+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:57:33.489346+0000","last_undegraded":"2024-09-10T22:57:33.489346+0000","last_fullsized":"2024-09-10T22:57:33.489346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934203+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:57:33.934203+0000","last_peered":"2024-09-10T22:57:33.934203+0000","last_clean":"2024-09-10T22:57:33.934203+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:57:33.934203+0000","last_undegraded":"2024-09-10T22:57:33.934203+0000","last_fullsized":"2024-09-10T22:57:33.934203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232756+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:58:33.232756+0000","last_peered":"2024-09-10T22:58:33.232756+0000","last_clean":"2024-09-10T22:58:33.232756+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:58:33.232756+0000","last_undegraded":"2024-09-10T22:58:33.232756+0000","last_fullsized":"2024-09-10T22:58:33.232756+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233102+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:58:33.233102+0000","last_peered":"2024-09-10T22:58:33.233102+0000","last_clean":"2024-09-10T22:58:33.233102+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:58:33.233102+0000","last_undegraded":"2024-09-10T22:58:33.233102+0000","last_fullsized":"2024-09-10T22:58:33.233102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233243+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:58:33.233243+0000","last_peered":"2024-09-10T22:58:33.233243+0000","last_clean":"2024-09-10T22:58:33.233243+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:58:33.233243+0000","last_undegraded":"2024-09-10T22:58:33.233243+0000","last_fullsized":"2024-09-10T22:58:33.233243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183267+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:58:33.183267+0000","last_peered":"2024-09-10T22:58:33.183267+0000","last_clean":"2024-09-10T22:58:33.183267+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:58:33.183267+0000","last_undegraded":"2024-09-10T22:58:33.183267+0000","last_fullsized":"2024-09-10T22:58:33.183267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183093+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:58:33.183093+0000","last_peered":"2024-09-10T22:58:33.183093+0000","last_clean":"2024-09-10T22:58:33.183093+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:58:33.183093+0000","last_undegraded":"2024-09-10T22:58:33.183093+0000","last_fullsized":"2024-09-10T22:58:33.183093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233085+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:58:33.233085+0000","last_peered":"2024-09-10T22:58:33.233085+0000","last_clean":"2024-09-10T22:58:33.233085+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:58:33.233085+0000","last_undegraded":"2024-09-10T22:58:33.233085+0000","last_fullsized":"2024-09-10T22:58:33.233085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184318+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:58:33.184318+0000","last_peered":"2024-09-10T22:58:33.184318+0000","last_clean":"2024-09-10T22:58:33.184318+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:58:33.184318+0000","last_undegraded":"2024-09-10T22:58:33.184318+0000","last_fullsized":"2024-09-10T22:58:33.184318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492168+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:57:33.492168+0000","last_peered":"2024-09-10T22:57:33.492168+0000","last_clean":"2024-09-10T22:57:33.492168+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:57:33.492168+0000","last_undegraded":"2024-09-10T22:57:33.492168+0000","last_fullsized":"2024-09-10T22:57:33.492168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233325+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:58:33.233325+0000","last_peered":"2024-09-10T22:58:33.233325+0000","last_clean":"2024-09-10T22:58:33.233325+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:58:33.233325+0000","last_undegraded":"2024-09-10T22:58:33.233325+0000","last_fullsized":"2024-09-10T22:58:33.233325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636233+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:57:33.636233+0000","last_peered":"2024-09-10T22:57:33.636233+0000","last_clean":"2024-09-10T22:57:33.636233+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:57:33.636233+0000","last_undegraded":"2024-09-10T22:57:33.636233+0000","last_fullsized":"2024-09-10T22:57:33.636233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.181712+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:58:33.181712+0000","last_peered":"2024-09-10T22:58:33.181712+0000","last_clean":"2024-09-10T22:58:33.181712+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:58:33.181712+0000","last_undegraded":"2024-09-10T22:58:33.181712+0000","last_fullsized":"2024-09-10T22:58:33.181712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.481869+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:58:32.481869+0000","last_peered":"2024-09-10T22:58:32.481869+0000","last_clean":"2024-09-10T22:58:32.481869+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:58:32.481869+0000","last_undegraded":"2024-09-10T22:58:32.481869+0000","last_fullsized":"2024-09-10T22:58:32.481869+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495716+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:57:33.495716+0000","last_peered":"2024-09-10T22:57:33.495716+0000","last_clean":"2024-09-10T22:57:33.495716+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:57:33.495716+0000","last_undegraded":"2024-09-10T22:57:33.495716+0000","last_fullsized":"2024-09-10T22:57:33.495716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182007+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:58:33.182007+0000","last_peered":"2024-09-10T22:58:33.182007+0000","last_clean":"2024-09-10T22:58:33.182007+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:58:33.182007+0000","last_undegraded":"2024-09-10T22:58:33.182007+0000","last_fullsized":"2024-09-10T22:58:33.182007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184392+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:58:33.184392+0000","last_peered":"2024-09-10T22:58:33.184392+0000","last_clean":"2024-09-10T22:58:33.184392+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:58:33.184392+0000","last_undegraded":"2024-09-10T22:58:33.184392+0000","last_fullsized":"2024-09-10T22:58:33.184392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.968470+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:57:33.968470+0000","last_peered":"2024-09-10T22:57:33.968470+0000","last_clean":"2024-09-10T22:57:33.968470+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:57:33.968470+0000","last_undegraded":"2024-09-10T22:57:33.968470+0000","last_fullsized":"2024-09-10T22:57:33.968470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636127+0000","last_change":"2024-09-10T22:56:27.955659+0000","last_active":"2024-09-10T22:57:33.636127+0000","last_peered":"2024-09-10T22:57:33.636127+0000","last_clean":"2024-09-10T22:57:33.636127+0000","last_became_active":"2024-09-10T22:56:27.955259+0000","last_became_peered":"2024-09-10T22:56:27.955259+0000","last_unstale":"2024-09-10T22:57:33.636127+0000","last_undegraded":"2024-09-10T22:57:33.636127+0000","last_fullsized":"2024-09-10T22:57:33.636127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:25.558781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232505+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:58:33.232505+0000","last_peered":"2024-09-10T22:58:33.232505+0000","last_clean":"2024-09-10T22:58:33.232505+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:58:33.232505+0000","last_undegraded":"2024-09-10T22:58:33.232505+0000","last_fullsized":"2024-09-10T22:58:33.232505+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:27.586585+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:58:27.586585+0000","last_peered":"2024-09-10T22:58:27.586585+0000","last_clean":"2024-09-10T22:58:27.586585+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:58:27.586585+0000","last_undegraded":"2024-09-10T22:58:27.586585+0000","last_fullsized":"2024-09-10T22:58:27.586585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485425+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:57:33.485425+0000","last_peered":"2024-09-10T22:57:33.485425+0000","last_clean":"2024-09-10T22:57:33.485425+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:57:33.485425+0000","last_undegraded":"2024-09-10T22:57:33.485425+0000","last_fullsized":"2024-09-10T22:57:33.485425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232849+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:58:33.232849+0000","last_peered":"2024-09-10T22:58:33.232849+0000","last_clean":"2024-09-10T22:58:33.232849+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:58:33.232849+0000","last_undegraded":"2024-09-10T22:58:33.232849+0000","last_fullsized":"2024-09-10T22:58:33.232849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183879+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:58:33.183879+0000","last_peered":"2024-09-10T22:58:33.183879+0000","last_clean":"2024-09-10T22:58:33.183879+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:58:33.183879+0000","last_undegraded":"2024-09-10T22:58:33.183879+0000","last_fullsized":"2024-09-10T22:58:33.183879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488302+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:57:33.488302+0000","last_peered":"2024-09-10T22:57:33.488302+0000","last_clean":"2024-09-10T22:57:33.488302+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:57:33.488302+0000","last_undegraded":"2024-09-10T22:57:33.488302+0000","last_fullsized":"2024-09-10T22:57:33.488302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635806+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:57:33.635806+0000","last_peered":"2024-09-10T22:57:33.635806+0000","last_clean":"2024-09-10T22:57:33.635806+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:57:33.635806+0000","last_undegraded":"2024-09-10T22:57:33.635806+0000","last_fullsized":"2024-09-10T22:57:33.635806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184067+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:58:33.184067+0000","last_peered":"2024-09-10T22:58:33.184067+0000","last_clean":"2024-09-10T22:58:33.184067+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:58:33.184067+0000","last_undegraded":"2024-09-10T22:58:33.184067+0000","last_fullsized":"2024-09-10T22:58:33.184067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183801+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:58:33.183801+0000","last_peered":"2024-09-10T22:58:33.183801+0000","last_clean":"2024-09-10T22:58:33.183801+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:58:33.183801+0000","last_undegraded":"2024-09-10T22:58:33.183801+0000","last_fullsized":"2024-09-10T22:58:33.183801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495545+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:57:33.495545+0000","last_peered":"2024-09-10T22:57:33.495545+0000","last_clean":"2024-09-10T22:57:33.495545+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:57:33.495545+0000","last_undegraded":"2024-09-10T22:57:33.495545+0000","last_fullsized":"2024-09-10T22:57:33.495545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":233,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:30.592138+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:58:30.592138+0000","last_peered":"2024-09-10T22:58:30.592138+0000","last_clean":"2024-09-10T22:58:30.592138+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:58:30.592138+0000","last_undegraded":"2024-09-10T22:58:30.592138+0000","last_fullsized":"2024-09-10T22:58:30.592138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485135+0000","last_change":"2024-09-10T22:56:27.958158+0000","last_active":"2024-09-10T22:57:33.485135+0000","last_peered":"2024-09-10T22:57:33.485135+0000","last_clean":"2024-09-10T22:57:33.485135+0000","last_became_active":"2024-09-10T22:56:27.953944+0000","last_became_peered":"2024-09-10T22:56:27.953944+0000","last_unstale":"2024-09-10T22:57:33.485135+0000","last_undegraded":"2024-09-10T22:57:33.485135+0000","last_fullsized":"2024-09-10T22:57:33.485135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:31.335607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183450+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:58:33.183450+0000","last_peered":"2024-09-10T22:58:33.183450+0000","last_clean":"2024-09-10T22:58:33.183450+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:58:33.183450+0000","last_undegraded":"2024-09-10T22:58:33.183450+0000","last_fullsized":"2024-09-10T22:58:33.183450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184111+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:58:33.184111+0000","last_peered":"2024-09-10T22:58:33.184111+0000","last_clean":"2024-09-10T22:58:33.184111+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:58:33.184111+0000","last_undegraded":"2024-09-10T22:58:33.184111+0000","last_fullsized":"2024-09-10T22:58:33.184111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933615+0000","last_change":"2024-09-10T22:56:27.961979+0000","last_active":"2024-09-10T22:57:33.933615+0000","last_peered":"2024-09-10T22:57:33.933615+0000","last_clean":"2024-09-10T22:57:33.933615+0000","last_became_active":"2024-09-10T22:56:27.961354+0000","last_became_peered":"2024-09-10T22:56:27.961354+0000","last_unstale":"2024-09-10T22:57:33.933615+0000","last_undegraded":"2024-09-10T22:57:33.933615+0000","last_fullsized":"2024-09-10T22:57:33.933615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:52.365127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492566+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:57:33.492566+0000","last_peered":"2024-09-10T22:57:33.492566+0000","last_clean":"2024-09-10T22:57:33.492566+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:57:33.492566+0000","last_undegraded":"2024-09-10T22:57:33.492566+0000","last_fullsized":"2024-09-10T22:57:33.492566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233135+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:58:33.233135+0000","last_peered":"2024-09-10T22:58:33.233135+0000","last_clean":"2024-09-10T22:58:33.233135+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:58:33.233135+0000","last_undegraded":"2024-09-10T22:58:33.233135+0000","last_fullsized":"2024-09-10T22:58:33.233135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484599+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:57:33.484599+0000","last_peered":"2024-09-10T22:57:33.484599+0000","last_clean":"2024-09-10T22:57:33.484599+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:57:33.484599+0000","last_undegraded":"2024-09-10T22:57:33.484599+0000","last_fullsized":"2024-09-10T22:57:33.484599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184129+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:58:33.184129+0000","last_peered":"2024-09-10T22:58:33.184129+0000","last_clean":"2024-09-10T22:58:33.184129+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:58:33.184129+0000","last_undegraded":"2024-09-10T22:58:33.184129+0000","last_fullsized":"2024-09-10T22:58:33.184129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:29.590216+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:58:29.590216+0000","last_peered":"2024-09-10T22:58:29.590216+0000","last_clean":"2024-09-10T22:58:29.590216+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:58:29.590216+0000","last_undegraded":"2024-09-10T22:58:29.590216+0000","last_fullsized":"2024-09-10T22:58:29.590216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185565+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:58:33.185565+0000","last_peered":"2024-09-10T22:58:33.185565+0000","last_clean":"2024-09-10T22:58:33.185565+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:58:33.185565+0000","last_undegraded":"2024-09-10T22:58:33.185565+0000","last_fullsized":"2024-09-10T22:58:33.185565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492234+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:57:33.492234+0000","last_peered":"2024-09-10T22:57:33.492234+0000","last_clean":"2024-09-10T22:57:33.492234+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:57:33.492234+0000","last_undegraded":"2024-09-10T22:57:33.492234+0000","last_fullsized":"2024-09-10T22:57:33.492234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.484686+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:57:33.484686+0000","last_peered":"2024-09-10T22:57:33.484686+0000","last_clean":"2024-09-10T22:57:33.484686+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:57:33.484686+0000","last_undegraded":"2024-09-10T22:57:33.484686+0000","last_fullsized":"2024-09-10T22:57:33.484686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182268+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:58:33.182268+0000","last_peered":"2024-09-10T22:58:33.182268+0000","last_clean":"2024-09-10T22:58:33.182268+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:58:33.182268+0000","last_undegraded":"2024-09-10T22:58:33.182268+0000","last_fullsized":"2024-09-10T22:58:33.182268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.488488+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:57:33.488488+0000","last_peered":"2024-09-10T22:57:33.488488+0000","last_clean":"2024-09-10T22:57:33.488488+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:57:33.488488+0000","last_undegraded":"2024-09-10T22:57:33.488488+0000","last_fullsized":"2024-09-10T22:57:33.488488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185428+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:58:33.185428+0000","last_peered":"2024-09-10T22:58:33.185428+0000","last_clean":"2024-09-10T22:58:33.185428+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:58:33.185428+0000","last_undegraded":"2024-09-10T22:58:33.185428+0000","last_fullsized":"2024-09-10T22:58:33.185428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635694+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:57:33.635694+0000","last_peered":"2024-09-10T22:57:33.635694+0000","last_clean":"2024-09-10T22:57:33.635694+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:57:33.635694+0000","last_undegraded":"2024-09-10T22:57:33.635694+0000","last_fullsized":"2024-09-10T22:57:33.635694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.489621+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:57:33.489621+0000","last_peered":"2024-09-10T22:57:33.489621+0000","last_clean":"2024-09-10T22:57:33.489621+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:57:33.489621+0000","last_undegraded":"2024-09-10T22:57:33.489621+0000","last_fullsized":"2024-09-10T22:57:33.489621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184392+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:58:33.184392+0000","last_peered":"2024-09-10T22:58:33.184392+0000","last_clean":"2024-09-10T22:58:33.184392+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:58:33.184392+0000","last_undegraded":"2024-09-10T22:58:33.184392+0000","last_fullsized":"2024-09-10T22:58:33.184392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182799+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:58:33.182799+0000","last_peered":"2024-09-10T22:58:33.182799+0000","last_clean":"2024-09-10T22:58:33.182799+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:58:33.182799+0000","last_undegraded":"2024-09-10T22:58:33.182799+0000","last_fullsized":"2024-09-10T22:58:33.182799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184498+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:58:33.184498+0000","last_peered":"2024-09-10T22:58:33.184498+0000","last_clean":"2024-09-10T22:58:33.184498+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:58:33.184498+0000","last_undegraded":"2024-09-10T22:58:33.184498+0000","last_fullsized":"2024-09-10T22:58:33.184498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494636+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:57:33.494636+0000","last_peered":"2024-09-10T22:57:33.494636+0000","last_clean":"2024-09-10T22:57:33.494636+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:57:33.494636+0000","last_undegraded":"2024-09-10T22:57:33.494636+0000","last_fullsized":"2024-09-10T22:57:33.494636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183261+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:58:33.183261+0000","last_peered":"2024-09-10T22:58:33.183261+0000","last_clean":"2024-09-10T22:58:33.183261+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:58:33.183261+0000","last_undegraded":"2024-09-10T22:58:33.183261+0000","last_fullsized":"2024-09-10T22:58:33.183261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.479094+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:58:32.479094+0000","last_peered":"2024-09-10T22:58:32.479094+0000","last_clean":"2024-09-10T22:58:32.479094+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:58:32.479094+0000","last_undegraded":"2024-09-10T22:58:32.479094+0000","last_fullsized":"2024-09-10T22:58:32.479094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182794+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:58:33.182794+0000","last_peered":"2024-09-10T22:58:33.182794+0000","last_clean":"2024-09-10T22:58:33.182794+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:58:33.182794+0000","last_undegraded":"2024-09-10T22:58:33.182794+0000","last_fullsized":"2024-09-10T22:58:33.182794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182445+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:58:33.182445+0000","last_peered":"2024-09-10T22:58:33.182445+0000","last_clean":"2024-09-10T22:58:33.182445+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:58:33.182445+0000","last_undegraded":"2024-09-10T22:58:33.182445+0000","last_fullsized":"2024-09-10T22:58:33.182445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492474+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:57:33.492474+0000","last_peered":"2024-09-10T22:57:33.492474+0000","last_clean":"2024-09-10T22:57:33.492474+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:57:33.492474+0000","last_undegraded":"2024-09-10T22:57:33.492474+0000","last_fullsized":"2024-09-10T22:57:33.492474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.636120+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:57:33.636120+0000","last_peered":"2024-09-10T22:57:33.636120+0000","last_clean":"2024-09-10T22:57:33.636120+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:57:33.636120+0000","last_undegraded":"2024-09-10T22:57:33.636120+0000","last_fullsized":"2024-09-10T22:57:33.636120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.483464+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:58:32.483464+0000","last_peered":"2024-09-10T22:58:32.483464+0000","last_clean":"2024-09-10T22:58:32.483464+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:58:32.483464+0000","last_undegraded":"2024-09-10T22:58:32.483464+0000","last_fullsized":"2024-09-10T22:58:32.483464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183388+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:58:33.183388+0000","last_peered":"2024-09-10T22:58:33.183388+0000","last_clean":"2024-09-10T22:58:33.183388+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:58:33.183388+0000","last_undegraded":"2024-09-10T22:58:33.183388+0000","last_fullsized":"2024-09-10T22:58:33.183388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.186002+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:58:33.186002+0000","last_peered":"2024-09-10T22:58:33.186002+0000","last_clean":"2024-09-10T22:58:33.186002+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:58:33.186002+0000","last_undegraded":"2024-09-10T22:58:33.186002+0000","last_fullsized":"2024-09-10T22:58:33.186002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.181962+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:58:33.181962+0000","last_peered":"2024-09-10T22:58:33.181962+0000","last_clean":"2024-09-10T22:58:33.181962+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:58:33.181962+0000","last_undegraded":"2024-09-10T22:58:33.181962+0000","last_fullsized":"2024-09-10T22:58:33.181962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185965+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:58:33.185965+0000","last_peered":"2024-09-10T22:58:33.185965+0000","last_clean":"2024-09-10T22:58:33.185965+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:58:33.185965+0000","last_undegraded":"2024-09-10T22:58:33.185965+0000","last_fullsized":"2024-09-10T22:58:33.185965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183282+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:58:33.183282+0000","last_peered":"2024-09-10T22:58:33.183282+0000","last_clean":"2024-09-10T22:58:33.183282+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:58:33.183282+0000","last_undegraded":"2024-09-10T22:58:33.183282+0000","last_fullsized":"2024-09-10T22:58:33.183282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635935+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:57:33.635935+0000","last_peered":"2024-09-10T22:57:33.635935+0000","last_clean":"2024-09-10T22:57:33.635935+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:57:33.635935+0000","last_undegraded":"2024-09-10T22:57:33.635935+0000","last_fullsized":"2024-09-10T22:57:33.635935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.480663+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:58:32.480663+0000","last_peered":"2024-09-10T22:58:32.480663+0000","last_clean":"2024-09-10T22:58:32.480663+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:58:32.480663+0000","last_undegraded":"2024-09-10T22:58:32.480663+0000","last_fullsized":"2024-09-10T22:58:32.480663+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.183478+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:58:33.183478+0000","last_peered":"2024-09-10T22:58:33.183478+0000","last_clean":"2024-09-10T22:58:33.183478+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:58:33.183478+0000","last_undegraded":"2024-09-10T22:58:33.183478+0000","last_fullsized":"2024-09-10T22:58:33.183478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.485363+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:57:33.485363+0000","last_peered":"2024-09-10T22:57:33.485363+0000","last_clean":"2024-09-10T22:57:33.485363+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:57:33.485363+0000","last_undegraded":"2024-09-10T22:57:33.485363+0000","last_fullsized":"2024-09-10T22:57:33.485363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.498868+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:58:32.498868+0000","last_peered":"2024-09-10T22:58:32.498868+0000","last_clean":"2024-09-10T22:58:32.498868+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:58:32.498868+0000","last_undegraded":"2024-09-10T22:58:32.498868+0000","last_fullsized":"2024-09-10T22:58:32.498868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.182419+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:58:33.182419+0000","last_peered":"2024-09-10T22:58:33.182419+0000","last_clean":"2024-09-10T22:58:33.182419+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:58:33.182419+0000","last_undegraded":"2024-09-10T22:58:33.182419+0000","last_fullsized":"2024-09-10T22:58:33.182419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491907+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:57:33.491907+0000","last_peered":"2024-09-10T22:57:33.491907+0000","last_clean":"2024-09-10T22:57:33.491907+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:57:33.491907+0000","last_undegraded":"2024-09-10T22:57:33.491907+0000","last_fullsized":"2024-09-10T22:57:33.491907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.491892+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:57:33.491892+0000","last_peered":"2024-09-10T22:57:33.491892+0000","last_clean":"2024-09-10T22:57:33.491892+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:57:33.491892+0000","last_undegraded":"2024-09-10T22:57:33.491892+0000","last_fullsized":"2024-09-10T22:57:33.491892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.635943+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:57:33.635943+0000","last_peered":"2024-09-10T22:57:33.635943+0000","last_clean":"2024-09-10T22:57:33.635943+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:57:33.635943+0000","last_undegraded":"2024-09-10T22:57:33.635943+0000","last_fullsized":"2024-09-10T22:57:33.635943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.934287+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:57:33.934287+0000","last_peered":"2024-09-10T22:57:33.934287+0000","last_clean":"2024-09-10T22:57:33.934287+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:57:33.934287+0000","last_undegraded":"2024-09-10T22:57:33.934287+0000","last_fullsized":"2024-09-10T22:57:33.934287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.233188+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:58:33.233188+0000","last_peered":"2024-09-10T22:58:33.233188+0000","last_clean":"2024-09-10T22:58:33.233188+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:58:33.233188+0000","last_undegraded":"2024-09-10T22:58:33.233188+0000","last_fullsized":"2024-09-10T22:58:33.233188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.490348+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:57:33.490348+0000","last_peered":"2024-09-10T22:57:33.490348+0000","last_clean":"2024-09-10T22:57:33.490348+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:57:33.490348+0000","last_undegraded":"2024-09-10T22:57:33.490348+0000","last_fullsized":"2024-09-10T22:57:33.490348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.184036+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:58:33.184036+0000","last_peered":"2024-09-10T22:58:33.184036+0000","last_clean":"2024-09-10T22:58:33.184036+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:58:33.184036+0000","last_undegraded":"2024-09-10T22:58:33.184036+0000","last_fullsized":"2024-09-10T22:58:33.184036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.185332+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:58:33.185332+0000","last_peered":"2024-09-10T22:58:33.185332+0000","last_clean":"2024-09-10T22:58:33.185332+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:58:33.185332+0000","last_undegraded":"2024-09-10T22:58:33.185332+0000","last_fullsized":"2024-09-10T22:58:33.185332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:58:33.232685+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:58:33.232685+0000","last_peered":"2024-09-10T22:58:33.232685+0000","last_clean":"2024-09-10T22:58:33.232685+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:58:33.232685+0000","last_undegraded":"2024-09-10T22:58:33.232685+0000","last_fullsized":"2024-09-10T22:58:33.232685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.495990+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:57:33.495990+0000","last_peered":"2024-09-10T22:57:33.495990+0000","last_clean":"2024-09-10T22:57:33.495990+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:57:33.495990+0000","last_undegraded":"2024-09-10T22:57:33.495990+0000","last_fullsized":"2024-09-10T22:57:33.495990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.494336+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:57:33.494336+0000","last_peered":"2024-09-10T22:57:33.494336+0000","last_clean":"2024-09-10T22:57:33.494336+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:57:33.494336+0000","last_undegraded":"2024-09-10T22:57:33.494336+0000","last_fullsized":"2024-09-10T22:57:33.494336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.933913+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:57:33.933913+0000","last_peered":"2024-09-10T22:57:33.933913+0000","last_clean":"2024-09-10T22:57:33.933913+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:57:33.933913+0000","last_undegraded":"2024-09-10T22:57:33.933913+0000","last_fullsized":"2024-09-10T22:57:33.933913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492318+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:57:33.492318+0000","last_peered":"2024-09-10T22:57:33.492318+0000","last_clean":"2024-09-10T22:57:33.492318+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:57:33.492318+0000","last_undegraded":"2024-09-10T22:57:33.492318+0000","last_fullsized":"2024-09-10T22:57:33.492318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:57:33.492424+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:57:33.492424+0000","last_peered":"2024-09-10T22:57:33.492424+0000","last_clean":"2024-09-10T22:57:33.492424+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:57:33.492424+0000","last_undegraded":"2024-09-10T22:57:33.492424+0000","last_fullsized":"2024-09-10T22:57:33.492424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T22:58:32.498657+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:58:32.498657+0000","last_peered":"2024-09-10T22:58:32.498657+0000","last_clean":"2024-09-10T22:58:32.498657+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:58:32.498657+0000","last_undegraded":"2024-09-10T22:58:32.498657+0000","last_fullsized":"2024-09-10T22:58:32.498657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":496,"num_read_kb":319,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":199140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612651,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":499051,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1148279,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":503271,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":962560,"data_stored":498983,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397570,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":1104921,"data_compressed":62490,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561096,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":494351,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1090733,"data_compressed":59507,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":499113,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":504321,"data_compressed":26633,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":557198,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":557736,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":495739,"data_compressed":23650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:58:35.733 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:58:35.733 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:58:35.733 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:58:35.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T22:58:36.117 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:36.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:36 smithi067 ceph-mon[30939]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:58:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:36 smithi092 ceph-mon[34223]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:58:36.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:36 smithi145 ceph-mon[34397]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 -- 172.21.15.67:0/2331418043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810a9b0 msgr2=0x7fa7e810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 --2- 172.21.15.67:0/2331418043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810a9b0 0x7fa7e810adf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009900 tx=0x7fa7d002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 -- 172.21.15.67:0/2331418043 shutdown_connections 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 --2- 172.21.15.67:0/2331418043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810b330 0x7fa7e8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 --2- 172.21.15.67:0/2331418043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810a9b0 0x7fa7e810adf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 --2- 172.21.15.67:0/2331418043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 -- 172.21.15.67:0/2331418043 >> 172.21.15.67:0/2331418043 conn(0x7fa7e8100a50 msgr2=0x7fa7e8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 -- 172.21.15.67:0/2331418043 shutdown_connections 2024-09-10T22:58:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.695+0000 7fa7ed800640 1 -- 172.21.15.67:0/2331418043 wait complete. 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.696+0000 7fa7ed800640 1 Processor -- start 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.696+0000 7fa7ed800640 1 -- start start 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.696+0000 7fa7ed800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.696+0000 7fa7ed800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810a9b0 0x7fa7e819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7ed800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7ed800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8114e10 con 0x7fa7e810b330 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7ed800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8114c90 con 0x7fa7e8104fa0 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7ed800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8114f90 con 0x7fa7e810a9b0 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60394/0 (socket says 172.21.15.67:60394) 2024-09-10T22:58:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e7400640 1 -- 172.21.15.67:0/468686159 learned_addr learned my addr 172.21.15.67:0/468686159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7de000640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810a9b0 0x7fa7e819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e7400640 1 -- 172.21.15.67:0/468686159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810a9b0 msgr2=0x7fa7e819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.697+0000 7fa7e7400640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810a9b0 0x7fa7e819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7e7400640 1 -- 172.21.15.67:0/468686159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 msgr2=0x7fa7e819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7e7400640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7e7400640 1 -- 172.21.15.67:0/468686159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e81a1410 con 0x7fa7e810b330 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7e6a00640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7e7400640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000eb10 tx=0x7fa7e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004b20 con 0x7fa7e810b330 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7e0004cc0 con 0x7fa7e810b330 2024-09-10T22:58:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81a1700 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.698+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81a7dc0 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.699+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0010690 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.700+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e8105380 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.702+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fa7e0005020 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.703+0000 7fa7dfe00640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 0x7fa7cc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.703+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa7e00e66b0 con 0x7fa7e810b330 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.703+0000 7fa7e6a00640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 0x7fa7cc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.704+0000 7fa7e6a00640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 0x7fa7cc0b77f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800bfd0 tx=0x7fa7d800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.708+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa7e00ad060 con 0x7fa7e810b330 2024-09-10T22:58:36.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.945+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa7e802e5c0 con 0x7fa7e810b330 2024-09-10T22:58:36.948 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:36.948 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:58:36.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.945+0000 7fa7dfe00640 1 -- 172.21.15.67:0/468686159 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa7e00ac960 con 0x7fa7e810b330 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 msgr2=0x7fa7cc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 0x7fa7cc0b77f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800bfd0 tx=0x7fa7d800b390 comp rx=0 tx=0).stop 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 msgr2=0x7fa7e81a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000eb10 tx=0x7fa7e000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 shutdown_connections 2024-09-10T22:58:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa7cc0b5330 0x7fa7cc0b77f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e810b330 0x7fa7e81a0c40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e810a9b0 0x7fa7e819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 --2- 172.21.15.67:0/468686159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7e8104fa0 0x7fa7e819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 >> 172.21.15.67:0/468686159 conn(0x7fa7e8100a50 msgr2=0x7fa7e8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 shutdown_connections 2024-09-10T22:58:36.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:36.948+0000 7fa7ed800640 1 -- 172.21.15.67:0/468686159 wait complete. 2024-09-10T22:58:37.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:37 smithi067 ceph-mon[30939]: pgmap v379: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-10T22:58:37.463 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:37 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/468686159' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:58:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:37 smithi092 ceph-mon[34223]: pgmap v379: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-10T22:58:37.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:37 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/468686159' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:58:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:37 smithi145 ceph-mon[34397]: pgmap v379: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-10T22:58:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:37 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/468686159' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:58:37.844 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:58:37.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:38.229 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 -- 172.21.15.67:0/2663161238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 msgr2=0x7f662810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 --2- 172.21.15.67:0/2663161238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662810f7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6620009ca0 tx=0x7f662002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 -- 172.21.15.67:0/2663161238 shutdown_connections 2024-09-10T22:58:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 --2- 172.21.15.67:0/2663161238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662810f7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 --2- 172.21.15.67:0/2663161238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 0x7f6628108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 --2- 172.21.15.67:0/2663161238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 -- 172.21.15.67:0/2663161238 >> 172.21.15.67:0/2663161238 conn(0x7f66280fe660 msgr2=0x7f6628100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.813+0000 7f6627e00640 1 -- 172.21.15.67:0/2663161238 shutdown_connections 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.814+0000 7f6627e00640 1 -- 172.21.15.67:0/2663161238 wait complete. 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.815+0000 7f6627e00640 1 Processor -- start 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.815+0000 7f6627e00640 1 -- start start 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.815+0000 7f6627e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 0x7f6628199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6628112b60 con 0x7f6628108fa0 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66281129e0 con 0x7f6628102c10 2024-09-10T22:58:38.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6628112ce0 con 0x7f6628108620 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60418/0 (socket says 172.21.15.67:60418) 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 -- 172.21.15.67:0/63766164 learned_addr learned my addr 172.21.15.67:0/63766164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6626000640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 0x7f6628199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6626a00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 -- 172.21.15.67:0/63766164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 msgr2=0x7f6628199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 0x7f6628199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 -- 172.21.15.67:0/63766164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 msgr2=0x7f6628198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6627400640 1 -- 172.21.15.67:0/63766164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662819eee0 con 0x7f6628108fa0 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.816+0000 7f6626a00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:38.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f6627400640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6620005cc0 tx=0x7f6620032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620032e00 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66200044d0 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620030b70 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662819f170 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.817+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66281a5890 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.818+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6628102ff0 con 0x7f6628108fa0 2024-09-10T22:58:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.821+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f6620004680 con 0x7f6628108fa0 2024-09-10T22:58:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.822+0000 7f661f400640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 0x7f660c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.822+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f66201075e0 con 0x7f6628108fa0 2024-09-10T22:58:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.822+0000 7f6626a00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 0x7f660c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.823+0000 7f6626a00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 0x7f660c0b78a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661800abe0 tx=0x7f661800a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:38.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:38.826+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66200cdf90 con 0x7f6628108fa0 2024-09-10T22:58:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.059+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66281a5bd0 con 0x7f6628108fa0 2024-09-10T22:58:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.059+0000 7f661f400640 1 -- 172.21.15.67:0/63766164 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f66200cd890 con 0x7f6628108fa0 2024-09-10T22:58:39.061 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:39.061 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:58:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 msgr2=0x7f660c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 0x7f660c0b78a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661800abe0 tx=0x7f661800a5c0 comp rx=0 tx=0).stop 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 msgr2=0x7f662819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6620005cc0 tx=0x7f6620032a10 comp rx=0 tx=0).stop 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 shutdown_connections 2024-09-10T22:58:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f660c0b53e0 0x7f660c0b78a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628108fa0 0x7f662819e710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6628108620 0x7f6628199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 --2- 172.21.15.67:0/63766164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6628102c10 0x7f6628198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.062+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 >> 172.21.15.67:0/63766164 conn(0x7f66280fe660 msgr2=0x7f66280fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.063+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 shutdown_connections 2024-09-10T22:58:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:39.063+0000 7f6627e00640 1 -- 172.21.15.67:0/63766164 wait complete. 2024-09-10T22:58:39.442 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:39 smithi067 ceph-mon[30939]: pgmap v380: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:39.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:39 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/63766164' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:39.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:39 smithi092 ceph-mon[34223]: pgmap v380: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:39.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:39 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/63766164' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:39 smithi145 ceph-mon[34397]: pgmap v380: 129 pgs: 2 activating, 127 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:39 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/63766164' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:39.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:40.186 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:40.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 -- 172.21.15.67:0/1234504316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7188102da0 msgr2=0x7f7188103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:40.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 --2- 172.21.15.67:0/1234504316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7188102da0 0x7f7188103180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f717c009900 tx=0x7f717c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 -- 172.21.15.67:0/1234504316 shutdown_connections 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 --2- 172.21.15.67:0/1234504316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188109130 0x7f718810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 --2- 172.21.15.67:0/1234504316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71881087b0 0x7f7188108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 --2- 172.21.15.67:0/1234504316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7188102da0 0x7f7188103180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 -- 172.21.15.67:0/1234504316 >> 172.21.15.67:0/1234504316 conn(0x7f71880fe810 msgr2=0x7f7188100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 -- 172.21.15.67:0/1234504316 shutdown_connections 2024-09-10T22:58:40.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.734+0000 7f7190000640 1 -- 172.21.15.67:0/1234504316 wait complete. 2024-09-10T22:58:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.735+0000 7f7190000640 1 Processor -- start 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.735+0000 7f7190000640 1 -- start start 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 0x7f718819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7188109130 0x7f71881a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7188112cf0 con 0x7f71881087b0 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7188112b70 con 0x7f7188102da0 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f7190000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7188112e70 con 0x7f7188109130 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f718cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f718cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59378/0 (socket says 172.21.15.67:59378) 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f718cc00640 1 -- 172.21.15.67:0/2025474587 learned_addr learned my addr 172.21.15.67:0/2025474587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f718e000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7188109130 0x7f71881a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.736+0000 7f718d600640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 0x7f718819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 -- 172.21.15.67:0/2025474587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7188109130 msgr2=0x7f71881a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7188109130 0x7f71881a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 -- 172.21.15.67:0/2025474587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 msgr2=0x7f718819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:40.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 0x7f718819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 -- 172.21.15.67:0/2025474587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71881a1430 con 0x7f71881087b0 2024-09-10T22:58:40.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718d600640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 0x7f718819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:40.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f718cc00640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f717400d950 tx=0x7f717400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174017070 con 0x7f71881087b0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71881a1720 con 0x7f71881087b0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7174009c30 con 0x7f71881087b0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.737+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400f620 con 0x7f71881087b0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.738+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71881a7fb0 con 0x7f71881087b0 2024-09-10T22:58:40.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.739+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7188104050 con 0x7f71881087b0 2024-09-10T22:58:40.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.741+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f7174010460 con 0x7f71881087b0 2024-09-10T22:58:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.743+0000 7f7186000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 0x7f71700b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.744+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f71740e56d0 con 0x7f71881087b0 2024-09-10T22:58:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.744+0000 7f718d600640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 0x7f71700b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.744+0000 7f718d600640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 0x7f71700b77f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f717c006010 tx=0x7f717c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:40.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.746+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f71740ac080 con 0x7f71881087b0 2024-09-10T22:58:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.982+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71881a8180 con 0x7f71881087b0 2024-09-10T22:58:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.983+0000 7f7186000640 1 -- 172.21.15.67:0/2025474587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f7174009de0 con 0x7f71881087b0 2024-09-10T22:58:40.985 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:40.985 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:58:40.985 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:58:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 msgr2=0x7f71700b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 0x7f71700b77f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f717c006010 tx=0x7f717c005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 msgr2=0x7f718819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f717400d950 tx=0x7f717400de10 comp rx=0 tx=0).stop 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 shutdown_connections 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f71700b5330 0x7f71700b77f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7188109130 0x7f71881a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71881087b0 0x7f718819b8c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 --2- 172.21.15.67:0/2025474587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7188102da0 0x7f718819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 >> 172.21.15.67:0/2025474587 conn(0x7f71880fe810 msgr2=0x7f71880ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 shutdown_connections 2024-09-10T22:58:40.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:40.986+0000 7f7190000640 1 -- 172.21.15.67:0/2025474587 wait complete. 2024-09-10T22:58:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:41 smithi067 ceph-mon[30939]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:41.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:41 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2025474587' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:41.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:41 smithi092 ceph-mon[34223]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:41.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:41 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2025474587' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:41 smithi145 ceph-mon[34397]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-10T22:58:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:41 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2025474587' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:41.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:58:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 -- 172.21.15.67:0/2860683289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40109130 msgr2=0x7f9a4010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 --2- 172.21.15.67:0/2860683289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40109130 0x7f9a4010f930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a3802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 -- 172.21.15.67:0/2860683289 shutdown_connections 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 --2- 172.21.15.67:0/2860683289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40109130 0x7f9a4010f930 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 --2- 172.21.15.67:0/2860683289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a401087b0 0x7f9a40108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 --2- 172.21.15.67:0/2860683289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 0x7f9a40103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 -- 172.21.15.67:0/2860683289 >> 172.21.15.67:0/2860683289 conn(0x7f9a400fe810 msgr2=0x7f9a40100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:42.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 -- 172.21.15.67:0/2860683289 shutdown_connections 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.842+0000 7f9a46400640 1 -- 172.21.15.67:0/2860683289 wait complete. 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.843+0000 7f9a46400640 1 Processor -- start 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.843+0000 7f9a46400640 1 -- start start 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 0x7f9a400754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40109130 0x7f9a40075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112cb0 con 0x7f9a401087b0 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112b30 con 0x7f9a40102da0 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a46400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112e30 con 0x7f9a40109130 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59398/0 (socket says 172.21.15.67:59398) 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a36a00640 1 -- 172.21.15.67:0/545504740 learned_addr learned my addr 172.21.15.67:0/545504740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:42.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a3f400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 0x7f9a400754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.844+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40109130 0x7f9a40075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 -- 172.21.15.67:0/545504740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40109130 msgr2=0x7f9a40075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40109130 0x7f9a40075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 -- 172.21.15.67:0/545504740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 msgr2=0x7f9a400754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 0x7f9a400754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 -- 172.21.15.67:0/545504740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401a7560 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a36a00640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d950 tx=0x7f9a2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28017070 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a28009c30 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800f620 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401a7850 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.845+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401a7d10 con 0x7f9a401087b0 2024-09-10T22:58:42.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.847+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40103180 con 0x7f9a401087b0 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.849+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f9a28010460 con 0x7f9a401087b0 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.851+0000 7f9a3cc00640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 0x7f9a240b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.851+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f9a280e5cd0 con 0x7f9a401087b0 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.851+0000 7f9a3f400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 0x7f9a240b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.852+0000 7f9a3f400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 0x7f9a240b77f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005fd0 tx=0x7f9a30009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:42.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:42.855+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9a28009de0 con 0x7f9a401087b0 2024-09-10T22:58:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.058+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a4002e5c0 con 0x7f9a401087b0 2024-09-10T22:58:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.059+0000 7f9a3cc00640 1 -- 172.21.15.67:0/545504740 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25221 (secure 0 0 0) 0x7f9a280ac030 con 0x7f9a401087b0 2024-09-10T22:58:43.061 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:43.061 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:58:34.175811+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:58:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 msgr2=0x7f9a240b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 0x7f9a240b77f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005fd0 tx=0x7f9a30009680 comp rx=0 tx=0).stop 2024-09-10T22:58:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 msgr2=0x7f9a400759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d950 tx=0x7f9a2800de10 comp rx=0 tx=0).stop 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 shutdown_connections 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a240b5330 0x7f9a240b77f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40109130 0x7f9a40075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a401087b0 0x7f9a400759e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 --2- 172.21.15.67:0/545504740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102da0 0x7f9a400754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.062+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 >> 172.21.15.67:0/545504740 conn(0x7f9a400fe810 msgr2=0x7f9a400ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.063+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 shutdown_connections 2024-09-10T22:58:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:43.063+0000 7f9a46400640 1 -- 172.21.15.67:0/545504740 wait complete. 2024-09-10T22:58:43.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:43 smithi067 ceph-mon[30939]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 574 B/s wr, 7 op/s 2024-09-10T22:58:43.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:43 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/545504740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:58:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:43 smithi092 ceph-mon[34223]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 574 B/s wr, 7 op/s 2024-09-10T22:58:43.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:43 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/545504740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:58:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:43 smithi145 ceph-mon[34397]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 574 B/s wr, 7 op/s 2024-09-10T22:58:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:43 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/545504740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:58:43.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 -- 172.21.15.67:0/2317717348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38108620 msgr2=0x7f8d38108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2317717348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38108620 0x7f8d38108a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009900 tx=0x7f8d2402a410 comp rx=0 tx=0).stop 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 -- 172.21.15.67:0/2317717348 shutdown_connections 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2317717348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108fa0 0x7f8d3810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2317717348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38108620 0x7f8d38108a60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2317717348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38102c10 0x7f8d38102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 -- 172.21.15.67:0/2317717348 >> 172.21.15.67:0/2317717348 conn(0x7f8d380fe660 msgr2=0x7f8d38100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 -- 172.21.15.67:0/2317717348 shutdown_connections 2024-09-10T22:58:44.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.927+0000 7f8d3d600640 1 -- 172.21.15.67:0/2317717348 wait complete. 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.928+0000 7f8d3d600640 1 Processor -- start 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 -- start start 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108620 0x7f8d380759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 0x7f8d38075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38112b60 con 0x7f8d38102c10 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381129e0 con 0x7f8d38108fa0 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d3d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38112ce0 con 0x7f8d38108620 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:44.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59416/0 (socket says 172.21.15.67:59416) 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.929+0000 7f8d37e00640 1 -- 172.21.15.67:0/2059277951 learned_addr learned my addr 172.21.15.67:0/2059277951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37400640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108620 0x7f8d380759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d3cc00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 0x7f8d38075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 -- 172.21.15.67:0/2059277951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108620 msgr2=0x7f8d380759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108620 0x7f8d380759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 -- 172.21.15.67:0/2059277951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 msgr2=0x7f8d38075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 0x7f8d38075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 -- 172.21.15.67:0/2059277951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d381a72d0 con 0x7f8d38102c10 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d3cc00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 0x7f8d38075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d37e00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c920 tx=0x7f8d2800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.930+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28007c70 con 0x7f8d38102c10 2024-09-10T22:58:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.931+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d381a75c0 con 0x7f8d38102c10 2024-09-10T22:58:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.931+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d28007e10 con 0x7f8d38102c10 2024-09-10T22:58:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.931+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d280186f0 con 0x7f8d38102c10 2024-09-10T22:58:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.931+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d381a7ad0 con 0x7f8d38102c10 2024-09-10T22:58:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.932+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d38102ff0 con 0x7f8d38102c10 2024-09-10T22:58:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.934+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f8d2800f450 con 0x7f8d38102c10 2024-09-10T22:58:44.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.935+0000 7f8d34c00640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 0x7f8d1c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:44.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.936+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f8d280e6670 con 0x7f8d38102c10 2024-09-10T22:58:44.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.936+0000 7f8d37400640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 0x7f8d1c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:44.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.937+0000 7f8d37400640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 0x7f8d1c0b78a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d24006010 tx=0x7f8d24005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:44.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:44.937+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d28014030 con 0x7f8d38102c10 2024-09-10T22:58:45.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.171+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d381a7f30 con 0x7f8d38102c10 2024-09-10T22:58:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.171+0000 7f8d34c00640 1 -- 172.21.15.67:0/2059277951 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f8d280ad0b0 con 0x7f8d38102c10 2024-09-10T22:58:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:58:45.173 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:58:45.173 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 msgr2=0x7f8d1c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 0x7f8d1c0b78a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d24006010 tx=0x7f8d24005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 msgr2=0x7f8d380754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c920 tx=0x7f8d2800cde0 comp rx=0 tx=0).stop 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 shutdown_connections 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d1c0b53e0 0x7f8d1c0b78a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38108fa0 0x7f8d38075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d38108620 0x7f8d380759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:45.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 --2- 172.21.15.67:0/2059277951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d38102c10 0x7f8d380754a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:45.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 >> 172.21.15.67:0/2059277951 conn(0x7f8d380fe660 msgr2=0x7f8d38101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:45.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 shutdown_connections 2024-09-10T22:58:45.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:45.174+0000 7f8d3d600640 1 -- 172.21.15.67:0/2059277951 wait complete. 2024-09-10T22:58:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:45 smithi092 ceph-mon[34223]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-10T22:58:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:45 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2059277951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:45 smithi067 ceph-mon[30939]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-10T22:58:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:45 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2059277951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:45 smithi145 ceph-mon[34397]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-10T22:58:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:45 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2059277951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:45.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:56:19.886591+0000', 'modified': '2024-09-10T22:57:14.572524+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34730, 'mds_1': 34691, 'mds_2': 34631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24664': {'gid': 24664, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/2757139462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 2757139462}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 2757139462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/1285378679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 1285378679}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 1285378679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24691': {'gid': 24691, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6839/4174586003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 4174586003}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 4174586003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34631': {'gid': 34631, 'name': 'k', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.92:6833/307002386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 307002386}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 307002386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34691': {'gid': 34691, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.92:6837/523163273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 523163273}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 523163273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34730': {'gid': 34730, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/1949986898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1949986898}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1949986898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34730, 'qdb_cluster': [34730, 34691, 34631]} 2024-09-10T22:58:45.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T22:58:45.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:58:46.364 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:46.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 -- 172.21.15.67:0/598324147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f0102dc0 msgr2=0x7fd6f01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 --2- 172.21.15.67:0/598324147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01031a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 -- 172.21.15.67:0/598324147 shutdown_connections 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 --2- 172.21.15.67:0/598324147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 0x7fd6f010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 --2- 172.21.15.67:0/598324147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f01087d0 0x7fd6f0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 --2- 172.21.15.67:0/598324147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01031a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 -- 172.21.15.67:0/598324147 >> 172.21.15.67:0/598324147 conn(0x7fd6f00fe830 msgr2=0x7fd6f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.945+0000 7fd6f7400640 1 -- 172.21.15.67:0/598324147 shutdown_connections 2024-09-10T22:58:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.946+0000 7fd6f7400640 1 -- 172.21.15.67:0/598324147 wait complete. 2024-09-10T22:58:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.946+0000 7fd6f7400640 1 Processor -- start 2024-09-10T22:58:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 -- start start 2024-09-10T22:58:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 0x7fd6f019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112cc0 con 0x7fd6f01087d0 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112b40 con 0x7fd6f0102dc0 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f7400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112e40 con 0x7fd6f0109150 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59434/0 (socket says 172.21.15.67:59434) 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6e7e00640 1 -- 172.21.15.67:0/151516843 learned_addr learned my addr 172.21.15.67:0/151516843 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.947+0000 7fd6f5400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 0x7fd6f019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 -- 172.21.15.67:0/151516843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 msgr2=0x7fd6f019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 0x7fd6f019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 -- 172.21.15.67:0/151516843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 msgr2=0x7fd6f01991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 -- 172.21.15.67:0/151516843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f019f2a0 con 0x7fd6f01087d0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800d950 tx=0x7fd6d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6ee000640 1 -- 172.21.15.67:0/151516843 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8017070 con 0x7fd6f01087d0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.948+0000 7fd6ee000640 1 -- 172.21.15.67:0/151516843 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6d8009c30 con 0x7fd6f01087d0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.949+0000 7fd6ee000640 1 -- 172.21.15.67:0/151516843 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800f620 con 0x7fd6f01087d0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.949+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f019f590 con 0x7fd6f01087d0 2024-09-10T22:58:46.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.949+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a5c50 con 0x7fd6f01087d0 2024-09-10T22:58:46.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 Processor -- start 2024-09-10T22:58:46.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 -- start start 2024-09-10T22:58:46.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d4259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d425f00 0x55fc2d4282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6ee000640 1 -- 172.21.15.67:0/151516843 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd6d8010460 con 0x7fd6f01087d0 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d38cd50 con 0x55fc2d419650 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d37c180 con 0x55fc2d425f00 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.952+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d37ff90 con 0x55fc2d419f70 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59450/0 (socket says 172.21.15.67:59450) 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f5400640 1 -- 172.21.15.67:0/4132865861 learned_addr learned my addr 172.21.15.67:0/4132865861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d4259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d425f00 0x55fc2d4282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 -- 172.21.15.67:0/4132865861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d425f00 msgr2=0x55fc2d4282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d425f00 0x55fc2d4282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 -- 172.21.15.67:0/4132865861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 msgr2=0x55fc2d419a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f4a00640 1 -- 172.21.15.67:0/4132865861 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2d346a70 con 0x55fc2d419f70 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.953+0000 7fd6f5400640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.954+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d4259c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00057c0 tx=0x7fd6e0032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8860194c18bb8b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.954+0000 7fd6ee000640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 0x7fd6d40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.954+0000 7fd6e5c00640 1 -- 172.21.15.67:0/4132865861 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e003f070 con 0x55fc2d419f70 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.954+0000 7fd6e5c00640 1 -- 172.21.15.67:0/4132865861 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6e00044c0 con 0x55fc2d419f70 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.954+0000 7fd6e5c00640 1 -- 172.21.15.67:0/4132865861 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035070 con 0x55fc2d419f70 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6ee000640 1 -- 172.21.15.67:0/151516843 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd6d80e5ca0 con 0x7fd6f01087d0 2024-09-10T22:58:46.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f5400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 0x7fd6d40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/4132865861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 msgr2=0x55fc2d4259c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d4259c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00057c0 tx=0x7fd6e0032c30 comp rx=0 tx=0).stop 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f5400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 0x7fd6d40b7870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005f40 tx=0x7fd6e8009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/4132865861 shutdown_connections 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d425f00 0x55fc2d4282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d4259c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/4132865861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d419650 0x55fc2d419a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.955+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/4132865861 >> 172.21.15.67:0/4132865861 conn(0x55fc2d417430 msgr2=0x55fc2d418620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.956+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/4132865861 shutdown_connections 2024-09-10T22:58:46.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.956+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/4132865861 wait complete. 2024-09-10T22:58:46.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.956+0000 7fd6f9ec4740 1 -- start start 2024-09-10T22:58:46.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.956+0000 7fd6f9ec4740 1 Processor -- start 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d422e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d4233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d1dd5f0 con 0x55fc2d425f00 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d317460 con 0x55fc2d419f70 2024-09-10T22:58:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d428bc0 con 0x55fc2d419650 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f9ec4740 10 client.0 _ll_register_callbacks cb 0x55fc2d327210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59464/0 (socket says 172.21.15.67:59464) 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f5400640 1 -- 172.21.15.67:0/334317201 learned_addr learned my addr 172.21.15.67:0/334317201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d422e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.957+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d4233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 -- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 msgr2=0x55fc2d4233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d4233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 -- 172.21.15.67:0/334317201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 msgr2=0x55fc2d423910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 -- 172.21.15.67:0/334317201 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2d0da540 con 0x55fc2d419650 2024-09-10T22:58:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f5400640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d422e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55fc2d418360 tx=0x7fd6e00053c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c85acc38e0c3917b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6de000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T22:58:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.958+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035070 con 0x55fc2d419650 2024-09-10T22:58:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc2d319a70 con 0x55fc2d419650 2024-09-10T22:58:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd6e0004ad0 con 0x55fc2d419650 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0055050 con 0x55fc2d419650 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6f9ec4740 10 client.25117 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc2d3346f0 con 0x55fc2d419650 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.959+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fc2d319a70 con 0x55fc2d419650 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e003b8c0 con 0x55fc2d419650 2024-09-10T22:58:46.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 10 client.25117 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 10 client.25117 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 20 client.25117 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4366b0 0x55fc2d437b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 4 client.25117 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc2d38eca0 con 0x55fc2d4366b0 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.960+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4366b0 0x55fc2d437b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.961+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4366b0 0x55fc2d437b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8e7df929a72694c server_cookie=107abe2292c6f007 in_seq=0 out_seq=0 2024-09-10T22:58:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.961+0000 7fd6de000640 10 client.25117 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:58:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.962+0000 7fd6de000640 1 -- 172.21.15.67:0/334317201 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc2d38eca0 con 0x55fc2d4366b0 2024-09-10T22:58:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.962+0000 7fd6de000640 10 client.25117 handle_command_reply: tid=0 2024-09-10T22:58:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.962+0000 7fd6f9ec4740 1 client.25117 shutdown 2024-09-10T22:58:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 msgr2=0x55fc2d422e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d422e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55fc2d418360 tx=0x7fd6e00053c0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 shutdown_connections 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4366b0 0x55fc2d437b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d4233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/334317201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d422e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6de000640 0 client.25117 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 >> 172.21.15.67:0/334317201 conn(0x55fc2d417430 msgr2=0x55fc2d412140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 shutdown_connections 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/334317201 wait complete. 2024-09-10T22:58:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.963+0000 7fd6f9ec4740 20 client.25117 trim_cache size 0 max 16384 2024-09-10T22:58:46.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.982+0000 7fd6f9ec4740 1 Processor -- start 2024-09-10T22:58:46.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.982+0000 7fd6f9ec4740 1 -- start start 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d436d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d40dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d435e60 con 0x55fc2d425f00 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d436530 con 0x55fc2d419f70 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d436c00 con 0x55fc2d419650 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59466/0 (socket says 172.21.15.67:59466) 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f5400640 1 -- 172.21.15.67:0/1811004641 learned_addr learned my addr 172.21.15.67:0/1811004641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d436d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d40dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f5400640 1 -- 172.21.15.67:0/1811004641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 msgr2=0x55fc2d436d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.983+0000 7fd6f5400640 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d436d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6f5400640 1 -- 172.21.15.67:0/1811004641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 msgr2=0x55fc2d40dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6f5400640 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d40dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6f5400640 1 -- 172.21.15.67:0/1811004641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2d428770 con 0x55fc2d425f00 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6f5400640 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8011fa0 tx=0x7fd6e800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1c5437734b8ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6e5200640 1 -- 172.21.15.67:0/1811004641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8003b70 con 0x55fc2d425f00 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6e5200640 1 -- 172.21.15.67:0/1811004641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6e8003d10 con 0x55fc2d425f00 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.984+0000 7fd6e5200640 1 -- 172.21.15.67:0/1811004641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e801b800 con 0x55fc2d425f00 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/1811004641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 msgr2=0x55fc2d423c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8011fa0 tx=0x7fd6e800b000 comp rx=0 tx=0).stop 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/1811004641 shutdown_connections 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d423c50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419f70 0x55fc2d40dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/1811004641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419650 0x55fc2d436d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/1811004641 >> 172.21.15.67:0/1811004641 conn(0x55fc2d417430 msgr2=0x55fc2d411920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/1811004641 shutdown_connections 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/1811004641 wait complete. 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 -- start start 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.985+0000 7fd6f9ec4740 1 Processor -- start 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 0x55fc2d43a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d43aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d43faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d4379a0 con 0x55fc2d425f00 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d4372d0 con 0x55fc2d419650 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc2d438070 con 0x55fc2d419f70 2024-09-10T22:58:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f9ec4740 10 client.0 _ll_register_callbacks cb 0x7fd6e0032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d43faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f4a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 0x55fc2d43a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d43faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59480/0 (socket says 172.21.15.67:59480) 2024-09-10T22:58:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6f5400640 1 -- 172.21.15.67:0/2131784515 learned_addr learned my addr 172.21.15.67:0/2131784515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.986+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d43aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 -- 172.21.15.67:0/2131784515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 msgr2=0x55fc2d43aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d43aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 -- 172.21.15.67:0/2131784515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 msgr2=0x55fc2d43faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d43faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 -- 172.21.15.67:0/2131784515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2d440190 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f4a00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 0x55fc2d43a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00057c0 tx=0x7fd6e0032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16367dcdce211ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6de000640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035030 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd6e00319f0 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.987+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e8003d10 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0066050 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 10 client.35153 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc2d4a3970 con 0x55fc2d419650 2024-09-10T22:58:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6e8003d10 con 0x55fc2d419650 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e0061030 con 0x55fc2d419650 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 10 client.35153 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 10 client.35153 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 20 client.35153 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4a84a0 0x55fc2d4a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 4 client.35153 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.988+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "status"}) -- 0x55fc2d38a750 con 0x55fc2d4a84a0 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.989+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4a84a0 0x55fc2d4a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.989+0000 7fd6e7e00640 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4a84a0 0x55fc2d4a71b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d191b19e221dbab server_cookie=55ba2e630d8f7026 in_seq=0 out_seq=0 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.989+0000 7fd6de000640 10 client.35153 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6de000640 1 -- 172.21.15.67:0/2131784515 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fc2d38a750 con 0x55fc2d4a84a0 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6de000640 10 client.35153 handle_command_reply: tid=0 2024-09-10T22:58:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 client.35153 shutdown 2024-09-10T22:58:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 msgr2=0x55fc2d43a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 0x55fc2d43a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00057c0 tx=0x7fd6e0032c30 comp rx=0 tx=0).stop 2024-09-10T22:58:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 shutdown_connections 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55fc2d4a84a0 0x55fc2d4a71b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fc2d425f00 0x55fc2d43faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc2d419f70 0x55fc2d43aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 --2- 172.21.15.67:0/2131784515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2d419650 0x55fc2d43a540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6de000640 0 client.35153 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 >> 172.21.15.67:0/2131784515 conn(0x55fc2d417430 msgr2=0x55fc2d412140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 shutdown_connections 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 1 -- 172.21.15.67:0/2131784515 wait complete. 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.990+0000 7fd6f9ec4740 20 client.35153 trim_cache size 0 max 16384 2024-09-10T22:58:46.994 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34631, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 96.576961940999993, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 195.98022535000001 2024-09-10T22:58:46.995 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 msgr2=0x7fd6d40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 0x7fd6d40b7870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005f40 tx=0x7fd6e8009600 comp rx=0 tx=0).stop 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 msgr2=0x7fd6f0199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800d950 tx=0x7fd6d800de10 comp rx=0 tx=0).stop 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 shutdown_connections 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd6d40b53b0 0x7fd6d40b7870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6f0109150 0x7fd6f019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6f01087d0 0x7fd6f0199730 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 --2- 172.21.15.67:0/151516843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6f0102dc0 0x7fd6f01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 >> 172.21.15.67:0/151516843 conn(0x7fd6f00fe830 msgr2=0x7fd6f0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 shutdown_connections 2024-09-10T22:58:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:46.991+0000 7fd6f7400640 1 -- 172.21.15.67:0/151516843 wait complete. 2024-09-10T22:58:47.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:47 smithi092 ceph-mon[34223]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s 2024-09-10T22:58:47.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:47 smithi067 ceph-mon[30939]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s 2024-09-10T22:58:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:47 smithi145 ceph-mon[34397]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s 2024-09-10T22:58:47.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:58:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 -- 172.21.15.67:0/3460822946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc86010b1a0 msgr2=0x7fc8601119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 --2- 172.21.15.67:0/3460822946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc86010b1a0 0x7fc8601119a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc858009ca0 tx=0x7fc85802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 -- 172.21.15.67:0/3460822946 shutdown_connections 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 --2- 172.21.15.67:0/3460822946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc86010b1a0 0x7fc8601119a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 --2- 172.21.15.67:0/3460822946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010a820 0x7fc86010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 --2- 172.21.15.67:0/3460822946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc860104e10 0x7fc8601051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 -- 172.21.15.67:0/3460822946 >> 172.21.15.67:0/3460822946 conn(0x7fc860100840 msgr2=0x7fc860102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 -- 172.21.15.67:0/3460822946 shutdown_connections 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.861+0000 7fc85f200640 1 -- 172.21.15.67:0/3460822946 wait complete. 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.862+0000 7fc85f200640 1 Processor -- start 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.862+0000 7fc85f200640 1 -- start start 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86010a820 0x7fc86019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010b1a0 0x7fc8601a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc860114d60 con 0x7fc860104e10 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc860114be0 con 0x7fc86010b1a0 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc860114ee0 con 0x7fc86010a820 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54764/0 (socket says 172.21.15.67:54764) 2024-09-10T22:58:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85de00640 1 -- 172.21.15.67:0/1944499443 learned_addr learned my addr 172.21.15.67:0/1944499443 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc85e800640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010b1a0 0x7fc8601a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.863+0000 7fc857e00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86010a820 0x7fc86019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 -- 172.21.15.67:0/1944499443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86010a820 msgr2=0x7fc86019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86010a820 0x7fc86019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 -- 172.21.15.67:0/1944499443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010b1a0 msgr2=0x7fc8601a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010b1a0 0x7fc8601a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 -- 172.21.15.67:0/1944499443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8601a11e0 con 0x7fc860104e10 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85de00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9f0 tx=0x7fc85000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc856a00640 1 -- 172.21.15.67:0/1944499443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000cd80 con 0x7fc860104e10 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8601a14d0 con 0x7fc860104e10 2024-09-10T22:58:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc856a00640 1 -- 172.21.15.67:0/1944499443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8500054b0 con 0x7fc860104e10 2024-09-10T22:58:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc856a00640 1 -- 172.21.15.67:0/1944499443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850010430 con 0x7fc860104e10 2024-09-10T22:58:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.864+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a7b90 con 0x7fc860104e10 2024-09-10T22:58:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.867+0000 7fc867632740 1 Processor -- start 2024-09-10T22:58:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 -- start start 2024-09-10T22:58:48.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc856a00640 1 -- 172.21.15.67:0/1944499443 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fc850005020 con 0x7fc860104e10 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffcc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffcf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5633bff37d50 con 0x5633bffc0400 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633bff27180 con 0x5633bffc0d20 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc867632740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633bff2af90 con 0x5633bffcccb0 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54770/0 (socket says 172.21.15.67:54770) 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.868+0000 7fc85e800640 1 -- 172.21.15.67:0/1568929303 learned_addr learned my addr 172.21.15.67:0/1568929303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85de00640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffcc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc857e00640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffcf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 -- 172.21.15.67:0/1568929303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 msgr2=0x5633bffcf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffcf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 -- 172.21.15.67:0/1568929303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 msgr2=0x5633bffcc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffcc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 -- 172.21.15.67:0/1568929303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bfef1a70 con 0x5633bffc0400 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85de00640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffcc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc85e800640 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc858009ca0 tx=0x7fc858031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1ec3a88b079ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.869+0000 7fc84f000640 1 -- 172.21.15.67:0/1568929303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858031980 con 0x5633bffc0400 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc84f000640 1 -- 172.21.15.67:0/1568929303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc858036070 con 0x5633bffc0400 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc84f000640 1 -- 172.21.15.67:0/1568929303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858004350 con 0x5633bffc0400 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc856a00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 0x7fc8440b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc856a00640 1 -- 172.21.15.67:0/1944499443 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fc8500e64e0 con 0x7fc860104e10 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc85de00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 0x7fc8440b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc867632740 1 -- 172.21.15.67:0/1568929303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 msgr2=0x5633bffc07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.870+0000 7fc867632740 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc858009ca0 tx=0x7fc858031450 comp rx=0 tx=0).stop 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 -- 172.21.15.67:0/1568929303 shutdown_connections 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffcf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffcc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 --2- 172.21.15.67:0/1568929303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc07e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 -- 172.21.15.67:0/1568929303 >> 172.21.15.67:0/1568929303 conn(0x5633bffbe230 msgr2=0x5633bffbf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 -- 172.21.15.67:0/1568929303 shutdown_connections 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 -- 172.21.15.67:0/1568929303 wait complete. 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc85de00640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 0x7fc8440b7920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8500ac490 tx=0x7fc850009720 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.871+0000 7fc867632740 1 -- start start 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 Processor -- start 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0d20 0x5633bffca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffcccb0 0x5633bffca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5633bfd885f0 con 0x5633bffc0400 2024-09-10T22:58:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633bfec2460 con 0x5633bffcccb0 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633bffcf820 con 0x5633bffc0d20 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc867632740 10 client.0 _ll_register_callbacks cb 0x5633bfed2210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.872+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54778/0 (socket says 172.21.15.67:54778) 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 -- 172.21.15.67:0/2111692124 learned_addr learned my addr 172.21.15.67:0/2111692124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc857e00640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0d20 0x5633bffca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffcccb0 0x5633bffca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 -- 172.21.15.67:0/2111692124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0d20 msgr2=0x5633bffca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0d20 0x5633bffca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 -- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffcccb0 msgr2=0x5633bffca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffcccb0 0x5633bffca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 -- 172.21.15.67:0/2111692124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bfc85540 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc85e800640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc858009620 tx=0x7fc858003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef832bab5b4729e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.873+0000 7fc843400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858050030 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fc858031d10 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633bfec4a70 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc85805d050 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc867632740 10 client.15366 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633bfedf6f0 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5633bfec4a70 con 0x5633bffc0400 2024-09-10T22:58:48.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.874+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc858043020 con 0x5633bffc0400 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 10 client.15366 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 10 client.15366 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 20 client.15366 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633bffdd490 0x5633bffde900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 4 client.15366 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633bff39ca0 con 0x5633bffdd490 2024-09-10T22:58:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.875+0000 7fc857e00640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633bffdd490 0x5633bffde900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.876+0000 7fc857e00640 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633bffdd490 0x5633bffde900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e059294ef9ecd473 server_cookie=10f528c95abd39cb in_seq=0 out_seq=0 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.876+0000 7fc843400640 10 client.15366 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.876+0000 7fc843400640 1 -- 172.21.15.67:0/2111692124 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633bff39ca0 con 0x5633bffdd490 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.876+0000 7fc843400640 10 client.15366 handle_command_reply: tid=0 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 client.15366 shutdown 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 msgr2=0x5633bffc9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc858009620 tx=0x7fc858003eb0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 shutdown_connections 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633bffdd490 0x5633bffde900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffcccb0 0x5633bffca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0d20 0x5633bffca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 --2- 172.21.15.67:0/2111692124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0400 0x5633bffc9cb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 >> 172.21.15.67:0/2111692124 conn(0x5633bffbe230 msgr2=0x5633bffc3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc843400640 0 client.15366 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 shutdown_connections 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 1 -- 172.21.15.67:0/2111692124 wait complete. 2024-09-10T22:58:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.877+0000 7fc867632740 20 client.15366 trim_cache size 0 max 16384 2024-09-10T22:58:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.896+0000 7fc867632740 1 Processor -- start 2024-09-10T22:58:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.896+0000 7fc867632740 1 -- start start 2024-09-10T22:58:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0400 0x5633bffde230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffc7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5633bffdcc40 con 0x5633bffc0d20 2024-09-10T22:58:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633bffdd310 con 0x5633bffc0400 2024-09-10T22:58:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc867632740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633bffdd9e0 con 0x5633bffcccb0 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54788/0 (socket says 172.21.15.67:54788) 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 -- 172.21.15.67:0/4033207134 learned_addr learned my addr 172.21.15.67:0/4033207134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc85e800640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffc7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc85e800640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0400 0x5633bffde230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 -- 172.21.15.67:0/4033207134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 msgr2=0x5633bffc7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffc7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 -- 172.21.15.67:0/4033207134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0400 msgr2=0x5633bffde230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0400 0x5633bffde230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc857e00640 1 -- 172.21.15.67:0/4033207134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bffbe060 con 0x5633bffc0d20 2024-09-10T22:58:48.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.897+0000 7fc85e800640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffc7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.898+0000 7fc857e00640 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc86010a3c0 tx=0x7fc84800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf1564a72ed6c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.898+0000 7fc84e600640 1 -- 172.21.15.67:0/4033207134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848015070 con 0x5633bffc0d20 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.898+0000 7fc84e600640 1 -- 172.21.15.67:0/4033207134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc848003f00 con 0x5633bffc0d20 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.898+0000 7fc84e600640 1 -- 172.21.15.67:0/4033207134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848004b20 con 0x5633bffc0d20 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- 172.21.15.67:0/4033207134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 msgr2=0x5633bffc6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc86010a3c0 tx=0x7fc84800bd80 comp rx=0 tx=0).stop 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- 172.21.15.67:0/4033207134 shutdown_connections 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffcccb0 0x5633bffc7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffc0d20 0x5633bffc6ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 --2- 172.21.15.67:0/4033207134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0400 0x5633bffde230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- 172.21.15.67:0/4033207134 >> 172.21.15.67:0/4033207134 conn(0x5633bffbe230 msgr2=0x5633bffbf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- 172.21.15.67:0/4033207134 shutdown_connections 2024-09-10T22:58:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- 172.21.15.67:0/4033207134 wait complete. 2024-09-10T22:58:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.899+0000 7fc867632740 1 -- start start 2024-09-10T22:58:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 Processor -- start 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0400 0x5633bffe1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffe20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5633bffde780 con 0x5633bffcccb0 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633bffde0b0 con 0x5633bffc0d20 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633bffdee50 con 0x5633bffc0400 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc867632740 10 client.0 _ll_register_callbacks cb 0x7fc858003af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.900+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54796/0 (socket says 172.21.15.67:54796) 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 -- 172.21.15.67:0/3503183030 learned_addr learned my addr 172.21.15.67:0/3503183030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc857e00640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffe20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0400 0x5633bffe1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 -- 172.21.15.67:0/3503183030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0400 msgr2=0x5633bffe1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0400 0x5633bffe1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 -- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 msgr2=0x5633bffe20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffe20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 -- 172.21.15.67:0/3503183030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bffbecb0 con 0x5633bffcccb0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc85e800640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc85802ade0 tx=0x7fc858003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6101b1d1e8b1972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.901+0000 7fc843400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858050040 con 0x5633bffcccb0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fc8580046d0 con 0x5633bffcccb0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633bff35750 con 0x5633bffcccb0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc858030980 con 0x5633bffcccb0 2024-09-10T22:58:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 10 client.15372 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633c004a5a0 con 0x5633bffcccb0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5633bff35750 con 0x5633bffcccb0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc858043070 con 0x5633bffcccb0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 10 client.15372 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.902+0000 7fc867632740 20 client.15372 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633c004eeb0 0x5633c004dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc867632740 4 client.15372 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "status"}) -- 0x5633bff32b80 con 0x5633c004eeb0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc857e00640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633c004eeb0 0x5633c004dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc857e00640 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633c004eeb0 0x5633c004dbc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8482224ee52e69a7 server_cookie=dc59f061b80514d8 in_seq=0 out_seq=0 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.903+0000 7fc843400640 10 client.15372 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:58:48.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc843400640 1 -- 172.21.15.67:0/3503183030 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5633bff32b80 con 0x5633c004eeb0 2024-09-10T22:58:48.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc843400640 10 client.15372 handle_command_reply: tid=0 2024-09-10T22:58:48.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 client.15372 shutdown 2024-09-10T22:58:48.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 msgr2=0x5633bffe6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc85802ade0 tx=0x7fc858003e40 comp rx=0 tx=0).stop 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 shutdown_connections 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5633c004eeb0 0x5633c004dbc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5633bffcccb0 0x5633bffe6a50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633bffc0d20 0x5633bffe20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 --2- 172.21.15.67:0/3503183030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633bffc0400 0x5633bffe1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc843400640 0 client.15372 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 >> 172.21.15.67:0/3503183030 conn(0x5633bffbe230 msgr2=0x5633bffcb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 shutdown_connections 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 1 -- 172.21.15.67:0/3503183030 wait complete. 2024-09-10T22:58:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.904+0000 7fc867632740 20 client.15372 trim_cache size 0 max 16384 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34691, 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 99.503368629999997, 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:58:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 176.69785299099999 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 msgr2=0x7fc8440b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 0x7fc8440b7920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8500ac490 tx=0x7fc850009720 comp rx=0 tx=0).stop 2024-09-10T22:58:48.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 msgr2=0x7fc86019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9f0 tx=0x7fc85000edf0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 shutdown_connections 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8440b5460 0x7fc8440b7920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc86010b1a0 0x7fc8601a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86010a820 0x7fc86019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 --2- 172.21.15.67:0/1944499443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc860104e10 0x7fc86019b130 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 >> 172.21.15.67:0/1944499443 conn(0x7fc860100840 msgr2=0x7fc860102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.905+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 shutdown_connections 2024-09-10T22:58:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:48.906+0000 7fc85f200640 1 -- 172.21.15.67:0/1944499443 wait complete. 2024-09-10T22:58:49.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:49 smithi092 ceph-mon[34223]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T22:58:49.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:49 smithi067 ceph-mon[30939]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T22:58:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:49 smithi145 ceph-mon[34397]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T22:58:49.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:58:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 -- 172.21.15.67:0/2390010871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 msgr2=0x7f49f8105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 --2- 172.21.15.67:0/2390010871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f8105380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009900 tx=0x7f49e802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 -- 172.21.15.67:0/2390010871 shutdown_connections 2024-09-10T22:58:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 --2- 172.21.15.67:0/2390010871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 0x7f49f8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 --2- 172.21.15.67:0/2390010871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 0x7f49f810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 --2- 172.21.15.67:0/2390010871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f8105380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 -- 172.21.15.67:0/2390010871 >> 172.21.15.67:0/2390010871 conn(0x7f49f8100a50 msgr2=0x7f49f8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 -- 172.21.15.67:0/2390010871 shutdown_connections 2024-09-10T22:58:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.659+0000 7f49fda00640 1 -- 172.21.15.67:0/2390010871 wait complete. 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 Processor -- start 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 -- start start 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 0x7f49f819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 0x7f49f81a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8114e40 con 0x7f49f8104fa0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8114cc0 con 0x7f49f810a9b0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.660+0000 7f49fda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8114fc0 con 0x7f49f810b330 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54814/0 (socket says 172.21.15.67:54814) 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 -- 172.21.15.67:0/3694480091 learned_addr learned my addr 172.21.15.67:0/3694480091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6000640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 0x7f49f819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f7400640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 0x7f49f81a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 -- 172.21.15.67:0/3694480091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 msgr2=0x7f49f81a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 0x7f49f81a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 -- 172.21.15.67:0/3694480091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 msgr2=0x7f49f819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 0x7f49f819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.661+0000 7f49f6a00640 1 -- 172.21.15.67:0/3694480091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a1520 con 0x7f49f8104fa0 2024-09-10T22:58:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49f6a00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f49e802a930 tx=0x7f49e8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49ef400640 1 -- 172.21.15.67:0/3694480091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e803f070 con 0x7f49f8104fa0 2024-09-10T22:58:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49ef400640 1 -- 172.21.15.67:0/3694480091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49e8003e40 con 0x7f49f8104fa0 2024-09-10T22:58:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a17b0 con 0x7f49f8104fa0 2024-09-10T22:58:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a7ed0 con 0x7f49f8104fa0 2024-09-10T22:58:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.662+0000 7f49ef400640 1 -- 172.21.15.67:0/3694480091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8035030 con 0x7f49f8104fa0 2024-09-10T22:58:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.665+0000 7f4a003fc740 1 Processor -- start 2024-09-10T22:58:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 -- start start 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f49ef400640 1 -- 172.21.15.67:0/3694480091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f49e8031400 con 0x7f49f8104fa0 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff3aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556f8fea3d50 con 0x556f8ff38af0 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556f8fe93180 con 0x556f8ff2cb60 2024-09-10T22:58:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f4a003fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556f8fe96f90 con 0x556f8ff2c240 2024-09-10T22:58:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f49f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff3aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f49f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff3aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54826/0 (socket says 172.21.15.67:54826) 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.666+0000 7f49f6000640 1 -- 172.21.15.67:0/2650611463 learned_addr learned my addr 172.21.15.67:0/2650611463 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f7400640 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 -- 172.21.15.67:0/2650611463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 msgr2=0x556f8ff2c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff2c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 -- 172.21.15.67:0/2650611463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 msgr2=0x556f8ff3aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff3aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 -- 172.21.15.67:0/2650611463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8fe5da70 con 0x556f8ff2cb60 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.667+0000 7f49f6a00640 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff385b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49e81041c0 tx=0x7f49e810adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71db28d28f3f4161 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.668+0000 7f49e3e00640 1 -- 172.21.15.67:0/2650611463 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8116070 con 0x556f8ff2cb60 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.668+0000 7f49e3e00640 1 -- 172.21.15.67:0/2650611463 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49e810fd70 con 0x556f8ff2cb60 2024-09-10T22:58:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.668+0000 7f49e3e00640 1 -- 172.21.15.67:0/2650611463 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8112db0 con 0x556f8ff2cb60 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.668+0000 7f49ef400640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 0x7f49dc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f49ef400640 1 -- 172.21.15.67:0/3694480091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f49e8106c30 con 0x7f49f8104fa0 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f49f7400640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 0x7f49dc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 -- 172.21.15.67:0/2650611463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 msgr2=0x556f8ff385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff385b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49e81041c0 tx=0x7f49e810adf0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 -- 172.21.15.67:0/2650611463 shutdown_connections 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff3aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff385b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 --2- 172.21.15.67:0/2650611463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff2c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 -- 172.21.15.67:0/2650611463 >> 172.21.15.67:0/2650611463 conn(0x556f8ff2a020 msgr2=0x556f8ff2b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 -- 172.21.15.67:0/2650611463 shutdown_connections 2024-09-10T22:58:50.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.669+0000 7f4a003fc740 1 -- 172.21.15.67:0/2650611463 wait complete. 2024-09-10T22:58:50.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.670+0000 7f49f7400640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 0x7f49dc0b77f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.670+0000 7f4a003fc740 1 -- start start 2024-09-10T22:58:50.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.670+0000 7f4a003fc740 1 Processor -- start 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff35fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff38af0 0x556f8ff364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556f8fcf45f0 con 0x556f8ff2c240 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556f8fe2e460 con 0x556f8ff2cb60 2024-09-10T22:58:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff3b560 con 0x556f8ff38af0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f4a003fc740 10 client.0 _ll_register_callbacks cb 0x556f8fe3e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54834/0 (socket says 172.21.15.67:54834) 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f49f6a00640 1 -- 172.21.15.67:0/434446369 learned_addr learned my addr 172.21.15.67:0/434446369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f49f6000640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff35fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.671+0000 7f49f7400640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff38af0 0x556f8ff364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 -- 172.21.15.67:0/434446369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff38af0 msgr2=0x556f8ff364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff38af0 0x556f8ff364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 -- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 msgr2=0x556f8ff35fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff35fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 -- 172.21.15.67:0/434446369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8fbf1540 con 0x556f8ff2c240 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49f6a00640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f49e80c94b0 tx=0x7f49e8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade95232aa8459ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8049020 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f49e80cd140 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f8fe30a70 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.672+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f49e8116070 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 10 client.15384 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f8fe4b6f0 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556f8fe30a70 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f49e80b40d0 con 0x556f8ff2c240 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 10 client.15384 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 20 client.15384 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ff49040 0x556f8ff4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 4 client.15384 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.673+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f8fea5ca0 con 0x556f8ff49040 2024-09-10T22:58:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.674+0000 7f49f6000640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ff49040 0x556f8ff4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.674+0000 7f49f6000640 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ff49040 0x556f8ff4a4b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778be767a5dcdeef server_cookie=a3ed993bf81ab4af in_seq=0 out_seq=0 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.674+0000 7f49e0c00640 10 client.15384 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.675+0000 7f49e0c00640 1 -- 172.21.15.67:0/434446369 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f8fea5ca0 con 0x556f8ff49040 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.675+0000 7f49e0c00640 10 client.15384 handle_command_reply: tid=0 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.675+0000 7f4a003fc740 1 client.15384 shutdown 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.675+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 msgr2=0x556f8ff35a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.675+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f49e80c94b0 tx=0x7f49e8005de0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 shutdown_connections 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ff49040 0x556f8ff4a4b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff38af0 0x556f8ff364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff35fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 --2- 172.21.15.67:0/434446369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2c240 0x556f8ff35a70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f49e0c00640 0 client.15384 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:58:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 >> 172.21.15.67:0/434446369 conn(0x556f8ff2a020 msgr2=0x556f8ff2fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 shutdown_connections 2024-09-10T22:58:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 1 -- 172.21.15.67:0/434446369 wait complete. 2024-09-10T22:58:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.676+0000 7f4a003fc740 20 client.15384 trim_cache size 0 max 16384 2024-09-10T22:58:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 Processor -- start 2024-09-10T22:58:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 -- start start 2024-09-10T22:58:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff32c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff487f0 con 0x556f8ff38af0 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff48ec0 con 0x556f8ff2cb60 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.695+0000 7f4a003fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff49590 con 0x556f8ff2c240 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54842/0 (socket says 172.21.15.67:54842) 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f7400640 1 -- 172.21.15.67:0/3869417873 learned_addr learned my addr 172.21.15.67:0/3869417873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff32c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6a00640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 -- 172.21.15.67:0/3869417873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 msgr2=0x556f8ff30250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff30250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 -- 172.21.15.67:0/3869417873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 msgr2=0x556f8ff331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f6000640 1 -- 172.21.15.67:0/3869417873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8fe4b6f0 con 0x556f8ff2cb60 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.696+0000 7f49f7400640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.697+0000 7f49f6000640 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff32c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x556f8ff4b630 tx=0x7f49e400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35097db5b0133250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.697+0000 7f49e3400640 1 -- 172.21.15.67:0/3869417873 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4003ce0 con 0x556f8ff2cb60 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.697+0000 7f49e3400640 1 -- 172.21.15.67:0/3869417873 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49e4003e80 con 0x556f8ff2cb60 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.697+0000 7f49e3400640 1 -- 172.21.15.67:0/3869417873 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4004e10 con 0x556f8ff2cb60 2024-09-10T22:58:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- 172.21.15.67:0/3869417873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 msgr2=0x556f8ff32c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff32c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x556f8ff4b630 tx=0x7f49e400b390 comp rx=0 tx=0).stop 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- 172.21.15.67:0/3869417873 shutdown_connections 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff38af0 0x556f8ff331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff2cb60 0x556f8ff32c80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 --2- 172.21.15.67:0/3869417873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff30250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- 172.21.15.67:0/3869417873 >> 172.21.15.67:0/3869417873 conn(0x556f8ff2a020 msgr2=0x556f8ff24ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- 172.21.15.67:0/3869417873 shutdown_connections 2024-09-10T22:58:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- 172.21.15.67:0/3869417873 wait complete. 2024-09-10T22:58:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.698+0000 7f4a003fc740 1 -- start start 2024-09-10T22:58:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 Processor -- start 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff52f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff38af0 0x556f8ff4d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff4a330 con 0x556f8ff2cb60 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff49c60 con 0x556f8ff38af0 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556f8ff4aa00 con 0x556f8ff2c240 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f4a003fc740 10 client.0 _ll_register_callbacks cb 0x7f49f0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.699+0000 7f49f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54858/0 (socket says 172.21.15.67:54858) 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 -- 172.21.15.67:0/1316329903 learned_addr learned my addr 172.21.15.67:0/1316329903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f7400640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff38af0 0x556f8ff4d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6a00640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff52f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 -- 172.21.15.67:0/1316329903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 msgr2=0x556f8ff52f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff52f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 -- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff38af0 msgr2=0x556f8ff4d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff38af0 0x556f8ff4d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 -- 172.21.15.67:0/1316329903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8ff30790 con 0x556f8ff2cb60 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49f6000640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005940 tx=0x7f49e4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a7227866f7f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.700+0000 7f49e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:58:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4029040 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f49e4003ae0 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f8fea1750 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f49e403a050 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f4a003fc740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f8ff29e50 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.701+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556f8fea1750 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f49e4035030 con 0x556f8ff2cb60 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 20 client.15390 populate_metadata read hostname 'smithi067' 2024-09-10T22:58:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ffc36f0 0x556f8ffc2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 4 client.15390 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:58:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "status"}) -- 0x556f8fe9eb80 con 0x556f8ffc36f0 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.702+0000 7f49f6a00640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ffc36f0 0x556f8ffc2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f49f6a00640 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ffc36f0 0x556f8ffc2400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a5fdba46d5351a server_cookie=5ffb553c5b46020a in_seq=0 out_seq=0 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f49e0c00640 10 client.15390 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f49e0c00640 1 -- 172.21.15.67:0/1316329903 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556f8fe9eb80 con 0x556f8ffc36f0 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f49e0c00640 10 client.15390 handle_command_reply: tid=0 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f4a003fc740 1 client.15390 shutdown 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 msgr2=0x556f8ff4d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.703+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005940 tx=0x7f49e4004180 comp rx=0 tx=0).stop 2024-09-10T22:58:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 shutdown_connections 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556f8ffc36f0 0x556f8ffc2400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556f8ff38af0 0x556f8ff4d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556f8ff2cb60 0x556f8ff4d3e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 --2- 172.21.15.67:0/1316329903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556f8ff2c240 0x556f8ff52f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f49e0c00640 0 client.15390 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 >> 172.21.15.67:0/1316329903 conn(0x556f8ff2a020 msgr2=0x556f8ff43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 shutdown_connections 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 1 -- 172.21.15.67:0/1316329903 wait complete. 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.704+0000 7f4a003fc740 20 client.15390 trim_cache size 0 max 16384 2024-09-10T22:58:50.709 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34730, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 150.80669560199999, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 163.15926003199999 2024-09-10T22:58:50.710 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 msgr2=0x7f49dc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 0x7f49dc0b77f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0009760 comp rx=0 tx=0).stop 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 msgr2=0x7f49f819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f49e802a930 tx=0x7f49e8003c20 comp rx=0 tx=0).stop 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 shutdown_connections 2024-09-10T22:58:50.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49dc0b5330 0x7f49dc0b77f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49f810b330 0x7f49f81a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49f810a9b0 0x7f49f819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 --2- 172.21.15.67:0/3694480091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49f8104fa0 0x7f49f819b470 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 >> 172.21.15.67:0/3694480091 conn(0x7f49f8100a50 msgr2=0x7f49f81014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 shutdown_connections 2024-09-10T22:58:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:50.705+0000 7f49fda00640 1 -- 172.21.15.67:0/3694480091 wait complete. 2024-09-10T22:58:51.524 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:51 smithi092 ceph-mon[34223]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:51.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:51 smithi067 ceph-mon[30939]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:51.629 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T22:58:51.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:58:51.639 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:58:51.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T22:58:51.640 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:51 smithi145 ceph-mon[34397]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:58:52.033 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 -- 172.21.15.67:0/3103825956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd4102dc0 msgr2=0x7f3dd41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 --2- 172.21.15.67:0/3103825956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd4102dc0 0x7f3dd41031a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009900 tx=0x7f3dc402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 -- 172.21.15.67:0/3103825956 shutdown_connections 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 --2- 172.21.15.67:0/3103825956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 0x7f3dd410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 --2- 172.21.15.67:0/3103825956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd41087d0 0x7f3dd4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 --2- 172.21.15.67:0/3103825956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd4102dc0 0x7f3dd41031a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 -- 172.21.15.67:0/3103825956 >> 172.21.15.67:0/3103825956 conn(0x7f3dd40fe830 msgr2=0x7f3dd4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 -- 172.21.15.67:0/3103825956 shutdown_connections 2024-09-10T22:58:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.629+0000 7f3dda400640 1 -- 172.21.15.67:0/3103825956 wait complete. 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.630+0000 7f3dda400640 1 Processor -- start 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.630+0000 7f3dda400640 1 -- start start 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 0x7f3dd40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 0x7f3dd4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112d10 con 0x7f3dd41087d0 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112b90 con 0x7f3dd4102dc0 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dda400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112e90 con 0x7f3dd4109150 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54876/0 (socket says 172.21.15.67:54876) 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 -- 172.21.15.67:0/2596368902 learned_addr learned my addr 172.21.15.67:0/2596368902 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd3400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 0x7f3dd40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd3e00640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 0x7f3dd4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 -- 172.21.15.67:0/2596368902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 msgr2=0x7f3dd4075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 0x7f3dd4075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 -- 172.21.15.67:0/2596368902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 msgr2=0x7f3dd40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 0x7f3dd40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd2a00640 1 -- 172.21.15.67:0/2596368902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd41a7600 con 0x7f3dd41087d0 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.631+0000 7f3dd3400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 0x7f3dd40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dd2a00640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000e9f0 tx=0x7f3dc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000cd80 con 0x7f3dd41087d0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dc00108c0 con 0x7f3dd41087d0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41a78f0 con 0x7f3dd41087d0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00187c0 con 0x7f3dd41087d0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.632+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41a7e80 con 0x7f3dd41087d0 2024-09-10T22:58:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.633+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd41031a0 con 0x7f3dd41087d0 2024-09-10T22:58:52.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.636+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f3dc0010430 con 0x7f3dd41087d0 2024-09-10T22:58:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.638+0000 7f3dcbe00640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 0x7f3db80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.638+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f3dc00e6420 con 0x7f3dd41087d0 2024-09-10T22:58:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.638+0000 7f3dd3400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 0x7f3db80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.639+0000 7f3dd3400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 0x7f3db80b7870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4006010 tx=0x7f3dc4000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.642+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3dc00ac780 con 0x7f3dd41087d0 2024-09-10T22:58:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.855+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3dd4063590 con 0x7f3db80b53b0 2024-09-10T22:58:52.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.856+0000 7f3dcbe00640 1 -- 172.21.15.67:0/2596368902 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3dd4063590 con 0x7f3db80b53b0 2024-09-10T22:58:52.858 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-09-10T22:58:52.858 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-09-10T22:58:52.858 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.858+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 msgr2=0x7f3db80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.858+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 0x7f3db80b7870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4006010 tx=0x7f3dc4000bc0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.858+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 msgr2=0x7f3dd40759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.858+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000e9f0 tx=0x7f3dc000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 shutdown_connections 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3db80b53b0 0x7f3db80b7870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd4109150 0x7f3dd4075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dd41087d0 0x7f3dd40759e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 --2- 172.21.15.67:0/2596368902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dd4102dc0 0x7f3dd40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 >> 172.21.15.67:0/2596368902 conn(0x7f3dd40fe830 msgr2=0x7f3dd4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 shutdown_connections 2024-09-10T22:58:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:52.859+0000 7f3dda400640 1 -- 172.21.15.67:0/2596368902 wait complete. 2024-09-10T22:58:53.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:53 smithi092 ceph-mon[34223]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:53.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:53 smithi092 ceph-mon[34223]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:53 smithi067 ceph-mon[30939]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:53 smithi067 ceph-mon[30939]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:53 smithi145 ceph-mon[34397]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:58:53.702 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:53 smithi145 ceph-mon[34397]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:53.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T22:58:54.163 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.753+0000 7ff314a00640 1 -- 172.21.15.67:0/4014021015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310101220 msgr2=0x7ff310101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.753+0000 7ff314a00640 1 --2- 172.21.15.67:0/4014021015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310101220 0x7ff310101600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 -- 172.21.15.67:0/4014021015 shutdown_connections 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 --2- 172.21.15.67:0/4014021015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310105ef0 0x7ff31010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 --2- 172.21.15.67:0/4014021015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 0x7ff3101059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 --2- 172.21.15.67:0/4014021015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310101220 0x7ff310101600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 -- 172.21.15.67:0/4014021015 >> 172.21.15.67:0/4014021015 conn(0x7ff310078100 msgr2=0x7ff3100ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 -- 172.21.15.67:0/4014021015 shutdown_connections 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.754+0000 7ff314a00640 1 -- 172.21.15.67:0/4014021015 wait complete. 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 Processor -- start 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 -- start start 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310101220 0x7ff310109920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 0x7ff310109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff310112ca0 con 0x7ff310105ef0 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff310112b20 con 0x7ff310101220 2024-09-10T22:58:54.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff314a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff310112e20 con 0x7ff310105570 2024-09-10T22:58:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.755+0000 7ff30e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54892/0 (socket says 172.21.15.67:54892) 2024-09-10T22:58:54.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 -- 172.21.15.67:0/530480328 learned_addr learned my addr 172.21.15.67:0/530480328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30de00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310101220 0x7ff310109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff305400640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 0x7ff310109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 -- 172.21.15.67:0/530480328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 msgr2=0x7ff310109e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 0x7ff310109e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 -- 172.21.15.67:0/530480328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310101220 msgr2=0x7ff310109920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310101220 0x7ff310109920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 -- 172.21.15.67:0/530480328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3101ab9b0 con 0x7ff310105ef0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.756+0000 7ff30e800640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff30800cce0 tx=0x7ff30800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.757+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800f450 con 0x7ff310105ef0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.757+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff30800fa80 con 0x7ff310105ef0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.757+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3101abca0 con 0x7ff310105ef0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.757+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018ad0 con 0x7ff310105ef0 2024-09-10T22:58:54.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.757+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3101ac190 con 0x7ff310105ef0 2024-09-10T22:58:54.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.758+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff310101600 con 0x7ff310105ef0 2024-09-10T22:58:54.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.760+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7ff30800f5f0 con 0x7ff310105ef0 2024-09-10T22:58:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.761+0000 7ff307400640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 0x7ff2f40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.762+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff3080e6830 con 0x7ff310105ef0 2024-09-10T22:58:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.762+0000 7ff30de00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 0x7ff2f40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.762+0000 7ff30de00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 0x7ff2f40b77f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3000057c0 tx=0x7ff3000058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.765+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff308014030 con 0x7ff310105ef0 2024-09-10T22:58:54.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.971+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff31010ec20 con 0x7ff2f40b5330 2024-09-10T22:58:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.984+0000 7ff307400640 1 -- 172.21.15.67:0/530480328 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7ff31010ec20 con 0x7ff2f40b5330 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (2m) 21s ago 2m 13.2M - 19.3.0-4854-g8f219c47 589dd18248e4 e464fcc390d9 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi092 running (2m) 30s ago 2m 25.1M - 19.3.0-4854-g8f219c47 589dd18248e4 dd372898bc1a 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi145 running (3m) 30s ago 3m 16.9M - 19.3.0-4854-g8f219c47 589dd18248e4 97b7f6d18f39 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (3m) 21s ago 3m 13.4M - 19.3.0-4854-g8f219c47 589dd18248e4 e0cac91f92f7 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi092 running (3m) 30s ago 3m 13.3M - 19.3.0-4854-g8f219c47 589dd18248e4 039bdd436f0d 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi145 running (3m) 30s ago 3m 17.0M - 19.3.0-4854-g8f219c47 589dd18248e4 8c448e44ed9b 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (2m) 21s ago 2m 13.2M - 19.3.0-4854-g8f219c47 589dd18248e4 784e4084029b 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi092 running (3m) 30s ago 3m 18.2M - 19.3.0-4854-g8f219c47 589dd18248e4 40f890febfb7 2024-09-10T22:58:54.986 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi145 running (2m) 30s ago 2m 18.1M - 19.3.0-4854-g8f219c47 589dd18248e4 20dc6ab074ec 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (3m) 21s ago 3m 13.5M - 19.3.0-4854-g8f219c47 589dd18248e4 683154f2e60c 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi092 running (3m) 30s ago 3m 17.8M - 19.3.0-4854-g8f219c47 589dd18248e4 ab7287038c30 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi145 running (3m) 30s ago 3m 13.6M - 19.3.0-4854-g8f219c47 589dd18248e4 7667c2a28a80 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283,8765 running (12m) 21s ago 12m 520M - 19.3.0-4854-g8f219c47 589dd18248e4 b9ff716034b7 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi092 *:8443,8765 running (10m) 30s ago 10m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 a10462e9fe10 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi145 *:8443,8765 running (10m) 30s ago 10m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 2ea72b5cedbb 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (12m) 21s ago 12m 55.3M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 f3b285da6f5d 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi092 running (10m) 30s ago 10m 40.4M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 7740611aba9b 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi145 running (10m) 30s ago 10m 42.0M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 325a515fb555 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (9m) 21s ago 9m 55.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 d75d82ba7a14 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi092 running (8m) 30s ago 8m 44.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 8042b7d66d11 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi145 running (8m) 30s ago 8m 46.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 ce21eb539f75 2024-09-10T22:58:54.987 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (7m) 21s ago 7m 55.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 effedfff4429 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi092 running (7m) 30s ago 7m 44.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 ae042f590cf5 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi145 running (6m) 30s ago 6m 43.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 7a96a474386a 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (6m) 21s ago 6m 42.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 8129aa5e7244 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi092 running (5m) 30s ago 5m 43.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f9f8e9bc3b6d 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi145 running (5m) 30s ago 5m 53.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 bca5441bd446 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (4m) 21s ago 4m 42.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 bc600a645283 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi092 running (4m) 30s ago 4m 42.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 2a978d1a292c 2024-09-10T22:58:54.988 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi145 running (3m) 30s ago 3m 52.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 d1d6a144db96 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 msgr2=0x7ff2f40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 0x7ff2f40b77f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3000057c0 tx=0x7ff3000058d0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 msgr2=0x7ff31010a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff30800cce0 tx=0x7ff30800a620 comp rx=0 tx=0).stop 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 shutdown_connections 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2f40b5330 0x7ff2f40b77f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff310105ef0 0x7ff31010a3a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff310105570 0x7ff310109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 --2- 172.21.15.67:0/530480328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff310101220 0x7ff310109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 >> 172.21.15.67:0/530480328 conn(0x7ff310078100 msgr2=0x7ff3101070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 shutdown_connections 2024-09-10T22:58:54.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:54.986+0000 7ff314a00640 1 -- 172.21.15.67:0/530480328 wait complete. 2024-09-10T22:58:55.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:55 smithi092 ceph-mon[34223]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:55 smithi067 ceph-mon[30939]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:55 smithi145 ceph-mon[34397]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:58:55.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T22:58:56.285 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:56.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:56 smithi067 ceph-mon[30939]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:56.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:56 smithi092 ceph-mon[34223]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:56.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:56 smithi145 ceph-mon[34397]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 -- 172.21.15.67:0/1057333432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c109150 msgr2=0x7efd9c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 --2- 172.21.15.67:0/1057333432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c109150 0x7efd9c10f950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd94009ca0 tx=0x7efd9402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 -- 172.21.15.67:0/1057333432 shutdown_connections 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 --2- 172.21.15.67:0/1057333432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c109150 0x7efd9c10f950 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 --2- 172.21.15.67:0/1057333432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c1087d0 0x7efd9c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 --2- 172.21.15.67:0/1057333432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 0x7efd9c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 -- 172.21.15.67:0/1057333432 >> 172.21.15.67:0/1057333432 conn(0x7efd9c0fe830 msgr2=0x7efd9c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.888+0000 7efda1800640 1 -- 172.21.15.67:0/1057333432 shutdown_connections 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.889+0000 7efda1800640 1 -- 172.21.15.67:0/1057333432 wait complete. 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.889+0000 7efda1800640 1 Processor -- start 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.889+0000 7efda1800640 1 -- start start 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 0x7efd9c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c109150 0x7efd9c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c112d10 con 0x7efd9c1087d0 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c112b90 con 0x7efd9c102dc0 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efda1800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c112e90 con 0x7efd9c109150 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efd9a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efd9a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54910/0 (socket says 172.21.15.67:54910) 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efd9a000640 1 -- 172.21.15.67:0/2711188031 learned_addr learned my addr 172.21.15.67:0/2711188031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 0x7efd9c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.890+0000 7efd9b400640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c109150 0x7efd9c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 -- 172.21.15.67:0/2711188031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c109150 msgr2=0x7efd9c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c109150 0x7efd9c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 -- 172.21.15.67:0/2711188031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 msgr2=0x7efd9c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 0x7efd9c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 -- 172.21.15.67:0/2711188031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c1a1390 con 0x7efd9c1087d0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd9a000640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c920 tx=0x7efd8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88007c70 con 0x7efd9c1087d0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd88007e10 con 0x7efd9c1087d0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88018710 con 0x7efd9c1087d0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c1a1680 con 0x7efd9c1087d0 2024-09-10T22:58:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.891+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c1a7d40 con 0x7efd9c1087d0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.892+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c1031a0 con 0x7efd9c1087d0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.895+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7efd8800f450 con 0x7efd9c1087d0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.896+0000 7efd93400640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 0x7efd800b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.896+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7efd880e6290 con 0x7efd9c1087d0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.896+0000 7efd9aa00640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 0x7efd800b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.897+0000 7efd9aa00640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 0x7efd800b77f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00bbe0 tx=0x7efd8c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:56.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:56.900+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efd88014030 con 0x7efd9c1087d0 2024-09-10T22:58:57.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.103+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efd9c063590 con 0x7efd800b5330 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.109+0000 7efd93400640 1 -- 172.21.15.67:0/2711188031 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efd9c063590 con 0x7efd800b5330 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 33s ago 3m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 33s ago 10m smithi067=x;smithi092=y;smithi145=z;count:3 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 33s ago 11m smithi067:172.21.15.67=a;smithi092:172.21.15.92=b;smithi145:172.21.15.145=c;count:3 2024-09-10T22:58:57.111 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 33s ago - 2024-09-10T22:58:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.111+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 msgr2=0x7efd800b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.111+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 0x7efd800b77f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00bbe0 tx=0x7efd8c00b390 comp rx=0 tx=0).stop 2024-09-10T22:58:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.111+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 msgr2=0x7efd9c19b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.111+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c920 tx=0x7efd8800cde0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 shutdown_connections 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd800b5330 0x7efd800b77f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9c109150 0x7efd9c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd9c1087d0 0x7efd9c19b820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 --2- 172.21.15.67:0/2711188031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c102dc0 0x7efd9c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 >> 172.21.15.67:0/2711188031 conn(0x7efd9c0fe830 msgr2=0x7efd9c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 shutdown_connections 2024-09-10T22:58:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:57.112+0000 7efda1800640 1 -- 172.21.15.67:0/2711188031 wait complete. 2024-09-10T22:58:57.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:57 smithi067 ceph-mon[30939]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:57 smithi092 ceph-mon[34223]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:57 smithi145 ceph-mon[34397]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T22:58:58.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T22:58:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:58.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:58 smithi092 ceph-mon[34223]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:58.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:58 smithi067 ceph-mon[30939]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:58.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:58 smithi145 ceph-mon[34397]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:58:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 -- 172.21.15.67:0/1585025021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca81087d0 msgr2=0x7f5ca8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 --2- 172.21.15.67:0/1585025021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca81087d0 0x7f5ca8108c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 -- 172.21.15.67:0/1585025021 shutdown_connections 2024-09-10T22:58:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 --2- 172.21.15.67:0/1585025021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca8109150 0x7f5ca810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 --2- 172.21.15.67:0/1585025021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca81087d0 0x7f5ca8108c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 --2- 172.21.15.67:0/1585025021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 -- 172.21.15.67:0/1585025021 >> 172.21.15.67:0/1585025021 conn(0x7f5ca80fe830 msgr2=0x7f5ca8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 -- 172.21.15.67:0/1585025021 shutdown_connections 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.975+0000 7f5cad200640 1 -- 172.21.15.67:0/1585025021 wait complete. 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 Processor -- start 2024-09-10T22:58:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 -- start start 2024-09-10T22:58:58.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca81087d0 0x7f5ca81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8109150 0x7f5ca819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8112cc0 con 0x7f5ca8102dc0 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8112b40 con 0x7f5ca8109150 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5cad200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8112e40 con 0x7f5ca81087d0 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.977+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57018/0 (socket says 172.21.15.67:57018) 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 -- 172.21.15.67:0/217797356 learned_addr learned my addr 172.21.15.67:0/217797356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca6800640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8109150 0x7f5ca819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:58.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5400640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca81087d0 0x7f5ca81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 -- 172.21.15.67:0/217797356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca81087d0 msgr2=0x7f5ca81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca81087d0 0x7f5ca81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 -- 172.21.15.67:0/217797356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8109150 msgr2=0x7f5ca819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8109150 0x7f5ca819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 -- 172.21.15.67:0/217797356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca819f260 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.978+0000 7f5ca5e00640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e9f0 tx=0x7f5c9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.979+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800cd80 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.979+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca819f550 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.979+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c980108c0 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.979+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980187c0 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.979+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a5de0 con 0x7f5ca8102dc0 2024-09-10T22:58:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.980+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca8104070 con 0x7f5ca8102dc0 2024-09-10T22:58:58.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.982+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f5c98010430 con 0x7f5ca8102dc0 2024-09-10T22:58:58.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.984+0000 7f5c9ea00640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 0x7f5c8c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:58.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.984+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f5c980e6990 con 0x7f5ca8102dc0 2024-09-10T22:58:58.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.984+0000 7f5ca5400640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 0x7f5c8c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:58.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.985+0000 7f5ca5400640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 0x7f5c8c0b7870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c94005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:58.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:58.985+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5c980ad340 con 0x7f5ca8102dc0 2024-09-10T22:58:59.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.186+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5ca8063590 con 0x7f5c8c0b53b0 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.188+0000 7f5c9ea00640 1 -- 172.21.15.67:0/217797356 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5ca8063590 con 0x7f5c8c0b53b0 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stdout:smithi092 172.21.15.92 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stdout:smithi145 172.21.15.145 2024-09-10T22:58:59.190 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 msgr2=0x7f5c8c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 0x7f5c8c0b7870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c94005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 msgr2=0x7f5ca81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e9f0 tx=0x7f5c9800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 shutdown_connections 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5c8c0b53b0 0x7f5c8c0b7870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca8109150 0x7f5ca819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ca81087d0 0x7f5ca81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 --2- 172.21.15.67:0/217797356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ca8102dc0 0x7f5ca81991b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 >> 172.21.15.67:0/217797356 conn(0x7f5ca80fe830 msgr2=0x7f5ca8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 shutdown_connections 2024-09-10T22:58:59.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:58:59.191+0000 7f5cad200640 1 -- 172.21.15.67:0/217797356 wait complete. 2024-09-10T22:58:59.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:58:59 smithi092 ceph-mon[34223]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:58:59 smithi067 ceph-mon[30939]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:58:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:58:59 smithi145 ceph-mon[34397]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:00.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T22:59:00.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:00 smithi092 ceph-mon[34223]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:00.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:00 smithi067 ceph-mon[30939]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:00.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:00 smithi145 ceph-mon[34397]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:01.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 -- 172.21.15.67:0/2493627389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 msgr2=0x7f665c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 --2- 172.21.15.67:0/2493627389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c10ac60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6648009900 tx=0x7f664802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 -- 172.21.15.67:0/2493627389 shutdown_connections 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 --2- 172.21.15.67:0/2493627389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c10b1a0 0x7f665c1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 --2- 172.21.15.67:0/2493627389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c10ac60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 --2- 172.21.15.67:0/2493627389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c104e10 0x7f665c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 -- 172.21.15.67:0/2493627389 >> 172.21.15.67:0/2493627389 conn(0x7f665c100840 msgr2=0x7f665c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 -- 172.21.15.67:0/2493627389 shutdown_connections 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.135+0000 7f665be00640 1 -- 172.21.15.67:0/2493627389 wait complete. 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.136+0000 7f665be00640 1 Processor -- start 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.136+0000 7f665be00640 1 -- start start 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c104e10 0x7f665c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c10b1a0 0x7f665c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f665c114cf0 con 0x7f665c10a820 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f665c114b70 con 0x7f665c10b1a0 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f665c114e70 con 0x7f665c104e10 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57040/0 (socket says 172.21.15.67:57040) 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 -- 172.21.15.67:0/2455972335 learned_addr learned my addr 172.21.15.67:0/2455972335 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665aa00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c104e10 0x7f665c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 -- 172.21.15.67:0/2455972335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c104e10 msgr2=0x7f665c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c104e10 0x7f665c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 -- 172.21.15.67:0/2455972335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c10b1a0 msgr2=0x7f665c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c10b1a0 0x7f665c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.137+0000 7f665a000640 1 -- 172.21.15.67:0/2455972335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c1a1150 con 0x7f665c10a820 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f665a000640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f664802a910 tx=0x7f6648003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664803f070 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c1a13e0 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6648003e40 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1a7b00 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.138+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648049060 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.139+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c1051f0 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.141+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f664804c020 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.144+0000 7f6653400640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 0x7f66400b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.144+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f66481069c0 con 0x7f665c10a820 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.144+0000 7f665aa00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 0x7f66400b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.145+0000 7f665aa00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 0x7f66400b78a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f664c005fd0 tx=0x7f664c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.148+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66480cd370 con 0x7f665c10a820 2024-09-10T22:59:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:01 smithi067 ceph-mon[30939]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:01.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.352+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f665c107e00 con 0x7f66400b53e0 2024-09-10T22:59:01.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.354+0000 7f6653400640 1 -- 172.21.15.67:0/2455972335 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f665c107e00 con 0x7f66400b53e0 2024-09-10T22:59:01.356 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T22:59:01.356 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T22:59:01.356 INFO:teuthology.orchestra.run.smithi067.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T22:59:01.357 INFO:teuthology.orchestra.run.smithi067.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 msgr2=0x7f66400b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 0x7f66400b78a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f664c005fd0 tx=0x7f664c005850 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 msgr2=0x7f665c19b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f664802a910 tx=0x7f6648003c20 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 shutdown_connections 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f66400b53e0 0x7f66400b78a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f665c10b1a0 0x7f665c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c10a820 0x7f665c19b5e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 --2- 172.21.15.67:0/2455972335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f665c104e10 0x7f665c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 >> 172.21.15.67:0/2455972335 conn(0x7f665c100840 msgr2=0x7f665c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 shutdown_connections 2024-09-10T22:59:01.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:01.357+0000 7f665be00640 1 -- 172.21.15.67:0/2455972335 wait complete. 2024-09-10T22:59:01.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:01 smithi092 ceph-mon[34223]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:01.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:01 smithi145 ceph-mon[34397]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:02.310 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:02 smithi067 ceph-mon[30939]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:02.349 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:59:02.359 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:59:02.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T22:59:02.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:02 smithi092 ceph-mon[34223]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:02.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:02 smithi145 ceph-mon[34397]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:02.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:03.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 -- 172.21.15.67:0/630549649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a4075e00 msgr2=0x7f85a410eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 --2- 172.21.15.67:0/630549649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a4075e00 0x7f85a410eca0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f858c009900 tx=0x7f858c02a410 comp rx=0 tx=0).stop 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 -- 172.21.15.67:0/630549649 shutdown_connections 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 --2- 172.21.15.67:0/630549649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 --2- 172.21.15.67:0/630549649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a4075e00 0x7f85a410eca0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 --2- 172.21.15.67:0/630549649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a40754e0 0x7f85a40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 -- 172.21.15.67:0/630549649 >> 172.21.15.67:0/630549649 conn(0x7f85a40fe680 msgr2=0x7f85a4100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 -- 172.21.15.67:0/630549649 shutdown_connections 2024-09-10T22:59:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.328+0000 7f85a9c00640 1 -- 172.21.15.67:0/630549649 wait complete. 2024-09-10T22:59:03.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.329+0000 7f85a9c00640 1 Processor -- start 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.329+0000 7f85a9c00640 1 -- start start 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a4075e00 0x7f85a419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4114c70 con 0x7f85a40754e0 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4114af0 con 0x7f85a410f1e0 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4114df0 con 0x7f85a4075e00 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57058/0 (socket says 172.21.15.67:57058) 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a3e00640 1 -- 172.21.15.67:0/2392587989 learned_addr learned my addr 172.21.15.67:0/2392587989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f859b400640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a4075e00 0x7f85a419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.330+0000 7f85a9200640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 -- 172.21.15.67:0/2392587989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a4075e00 msgr2=0x7f85a419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a4075e00 0x7f85a419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 -- 172.21.15.67:0/2392587989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 msgr2=0x7f85a41a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 -- 172.21.15.67:0/2392587989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a41a12f0 con 0x7f85a40754e0 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a9200640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a3e00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f859400eb90 tx=0x7f859400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594004d20 con 0x7f85a40754e0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f859400ee90 con 0x7f85a40754e0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594010690 con 0x7f85a40754e0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a41a15e0 con 0x7f85a40754e0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.331+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a7ca0 con 0x7f85a40754e0 2024-09-10T22:59:03.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.332+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a40758c0 con 0x7f85a40754e0 2024-09-10T22:59:03.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.336+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f8594010830 con 0x7f85a40754e0 2024-09-10T22:59:03.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.338+0000 7f85a1600640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 0x7f85880b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.338+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f85940e6a10 con 0x7f85a40754e0 2024-09-10T22:59:03.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.338+0000 7f859b400640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 0x7f85880b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.339+0000 7f859b400640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 0x7f85880b78a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f858c0057c0 tx=0x7f858c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:03.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.339+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8594056220 con 0x7f85a40754e0 2024-09-10T22:59:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:03 smithi092 ceph-mon[34223]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:03 smithi067 ceph-mon[30939]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:03.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.574+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f85a402e5c0 con 0x7f85a40754e0 2024-09-10T22:59:03.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.574+0000 7f85a1600640 1 -- 172.21.15.67:0/2392587989 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3082 (secure 0 0 0) 0x7f85940ad3c0 con 0x7f85a40754e0 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:e26 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:btime 2024-09-10T22:57:15:423621+0000 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 26 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-09-10T22:56:19.886591+0000 2024-09-10T22:59:03.577 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-09-10T22:57:14.572524+0000 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:max_xattr_size 65536 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 3 2024-09-10T22:59:03.578 INFO:teuthology.orchestra.run.smithi067.stdout:in 0,1,2 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=34730,1=34691,2=34631} 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:bal_rank_mask -1 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:qdb_cluster leader: 34730 members: 34730,34691,34631 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:34730} state up:active seq 5 addr [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{0:24691} state up:standby-replay seq 1 addr [v2:172.21.15.145:6838/4174586003,v1:172.21.15.145:6839/4174586003] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.579 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{1:34691} state up:active seq 21 addr [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{1:24676} state up:standby-replay seq 1 addr [v2:172.21.15.145:6836/1285378679,v1:172.21.15.145:6837/1285378679] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{2:34631} state up:active seq 26 addr [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{2:24664} state up:standby-replay seq 1 addr [v2:172.21.15.145:6834/2757139462,v1:172.21.15.145:6835/2757139462] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14787} state up:standby seq 1 addr [v2:172.21.15.67:6834/99494350,v1:172.21.15.67:6835/99494350] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{-1:14793} state up:standby seq 1 addr [v2:172.21.15.67:6836/1279733318,v1:172.21.15.67:6837/1279733318] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{-1:14802} state up:standby seq 1 addr [v2:172.21.15.92:6834/2446174169,v1:172.21.15.92:6835/2446174169] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{-1:14853} state up:standby seq 1 addr [v2:172.21.15.67:6838/4128597236,v1:172.21.15.67:6839/4128597236] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{-1:14865} state up:standby seq 1 addr [v2:172.21.15.67:6840/2410494517,v1:172.21.15.67:6841/2410494517] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.580 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{-1:24616} state up:standby seq 1 addr [v2:172.21.15.145:6832/482762321,v1:172.21.15.145:6833/482762321] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 msgr2=0x7f85880b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 0x7f85880b78a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f858c0057c0 tx=0x7f858c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 msgr2=0x7f85a419b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f859400eb90 tx=0x7f859400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 shutdown_connections 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85880b53e0 0x7f85880b78a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85a410f1e0 0x7f85a41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85a4075e00 0x7f85a419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 --2- 172.21.15.67:0/2392587989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a40754e0 0x7f85a419b240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.576+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 >> 172.21.15.67:0/2392587989 conn(0x7f85a40fe680 msgr2=0x7f85a40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.577+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 shutdown_connections 2024-09-10T22:59:03.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:03.577+0000 7f85a9c00640 1 -- 172.21.15.67:0/2392587989 wait complete. 2024-09-10T22:59:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:03 smithi145 ceph-mon[34397]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:04.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T22:59:04.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:04 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2392587989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:59:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:04 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2392587989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:59:04.753 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:04.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:04 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2392587989' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:59:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.372+0000 7f3c76200640 1 -- 172.21.15.67:0/1504912464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 msgr2=0x7f3c701135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.372+0000 7f3c76200640 1 --2- 172.21.15.67:0/1504912464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c701135e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 -- 172.21.15.67:0/1504912464 shutdown_connections 2024-09-10T22:59:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 --2- 172.21.15.67:0/1504912464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 0x7f3c70115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 --2- 172.21.15.67:0/1504912464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c701135e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 --2- 172.21.15.67:0/1504912464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c70075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 -- 172.21.15.67:0/1504912464 >> 172.21.15.67:0/1504912464 conn(0x7f3c700fe830 msgr2=0x7f3c70100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 -- 172.21.15.67:0/1504912464 shutdown_connections 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.373+0000 7f3c76200640 1 -- 172.21.15.67:0/1504912464 wait complete. 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 Processor -- start 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 -- start start 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c7019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 0x7f3c701a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701192b0 con 0x7f3c70075e60 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70119130 con 0x7f3c70075540 2024-09-10T22:59:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c76200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70119430 con 0x7f3c70113b20 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c6ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c6f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c7019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.374+0000 7f3c6ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57066/0 (socket says 172.21.15.67:57066) 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c7019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:48366/0 (socket says 172.21.15.67:48366) 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 -- 172.21.15.67:0/2033246301 learned_addr learned my addr 172.21.15.67:0/2033246301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6fe00640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 0x7f3c701a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 -- 172.21.15.67:0/2033246301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 msgr2=0x7f3c701a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 0x7f3c701a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 -- 172.21.15.67:0/2033246301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 msgr2=0x7f3c7019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c7019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 -- 172.21.15.67:0/2033246301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c701a5830 con 0x7f3c70075e60 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c6ea00640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c02a910 tx=0x7f3c5c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c03f070 con 0x7f3c70075e60 2024-09-10T22:59:05.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.375+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c5c003e40 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.376+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c035030 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.376+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c701a5ac0 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.376+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701ac1e0 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.377+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70075920 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.379+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f3c5c031b90 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.380+0000 7f3c67e00640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 0x7f3c540b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.381+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f3c5c106fa0 con 0x7f3c70075e60 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.381+0000 7f3c6f400640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 0x7f3c540b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.382+0000 7f3c6f400640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 0x7f3c540b77f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000abe0 tx=0x7f3c6000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.385+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3c5c0cd950 con 0x7f3c70075e60 2024-09-10T22:59:05.531 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:05 smithi145 ceph-mon[34397]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:05 smithi067 ceph-mon[30939]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:05.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.580+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3c701ac410 con 0x7f3c70075e60 2024-09-10T22:59:05.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.581+0000 7f3c67e00640 1 -- 172.21.15.67:0/2033246301 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+6076 (secure 0 0 0) 0x7f3c5c0cd250 con 0x7f3c70075e60 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 89 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-09-10T22:46:22.380818+0000 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-09-10T22:58:34.175811+0000 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2024-09-10T22:59:05.583 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release squid 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 9 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/3409065493,v1:172.21.15.67:6803/3409065493] [v2:172.21.15.67:6804/3409065493,v1:172.21.15.67:6805/3409065493] exists,up 7bebf858-8d3f-46bb-aea6-043414353290 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 14 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/1917308558,v1:172.21.15.92:6801/1917308558] [v2:172.21.15.92:6802/1917308558,v1:172.21.15.92:6803/1917308558] exists,up 002adf57-737e-4e91-9cf5-164895bffa8e 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 19 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/596498594,v1:172.21.15.145:6801/596498594] [v2:172.21.15.145:6802/596498594,v1:172.21.15.145:6803/596498594] exists,up 210c2de7-0ad3-4e05-b098-446542372b6a 2024-09-10T22:59:05.584 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 27 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/3883722074,v1:172.21.15.67:6811/3883722074] [v2:172.21.15.67:6812/3883722074,v1:172.21.15.67:6813/3883722074] exists,up 50b25a5d-e7a9-448a-b4ae-435368940e5d 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1079057674,v1:172.21.15.92:6809/1079057674] [v2:172.21.15.92:6810/1079057674,v1:172.21.15.92:6811/1079057674] exists,up 18c7f079-20a7-4726-bf63-797bd66fc512 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/510004066,v1:172.21.15.145:6809/510004066] [v2:172.21.15.145:6810/510004066,v1:172.21.15.145:6811/510004066] exists,up 8c623a06-702d-41dc-a7a7-56bc739d3199 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 44 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/2528830633,v1:172.21.15.67:6819/2528830633] [v2:172.21.15.67:6820/2528830633,v1:172.21.15.67:6821/2528830633] exists,up 9ca47089-cde1-479f-95ed-70c93dfafcf8 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 49 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/3770526101,v1:172.21.15.92:6817/3770526101] [v2:172.21.15.92:6818/3770526101,v1:172.21.15.92:6819/3770526101] exists,up 726d7795-37f9-452e-a9a0-d764920275a2 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 56 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/1197192386,v1:172.21.15.145:6817/1197192386] [v2:172.21.15.145:6818/1197192386,v1:172.21.15.145:6819/1197192386] exists,up f8630c3d-7812-4bca-a22d-3539f4fa03d9 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/3621288018,v1:172.21.15.67:6827/3621288018] [v2:172.21.15.67:6828/3621288018,v1:172.21.15.67:6829/3621288018] exists,up c52ab99b-433a-44a6-9fab-dbd35f1bf6d7 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 67 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/1466235369,v1:172.21.15.92:6825/1466235369] [v2:172.21.15.92:6826/1466235369,v1:172.21.15.92:6827/1466235369] exists,up e96f60cf-4ed2-4c62-bdfd-90c1632a5632 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/257658495,v1:172.21.15.145:6825/257658495] [v2:172.21.15.145:6826/257658495,v1:172.21.15.145:6827/257658495] exists,up d3c1830f-3c31-4764-92c2-ffcd6c11acce 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.1b [2,1] 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.1c [8,11] 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.1 [4,9] 2024-09-10T22:59:05.585 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.2 [3,9] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.c [5,8] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.10 [5,9] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.11 [4,7] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.1d [4,6] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.3a [4,8] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.e [5,8] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.12 [9,10] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.13 [9,8] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.16 [9,8] 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2682299290 expires 2024-09-11T22:48:31.958130+0000 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/363502482 expires 2024-09-11T22:48:31.958130+0000 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/2041533453 expires 2024-09-11T22:47:16.118298+0000 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3212905705 expires 2024-09-11T22:47:16.118298+0000 2024-09-10T22:59:05.586 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/82707279 expires 2024-09-11T22:46:47.684151+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3042343451 expires 2024-09-11T22:47:16.118298+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3255023915 expires 2024-09-11T22:46:47.684151+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2935874097 expires 2024-09-11T22:46:47.684151+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/294142637 expires 2024-09-11T22:46:47.684151+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/294142637 expires 2024-09-11T22:46:47.684151+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/2041533453 expires 2024-09-11T22:47:16.118298+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3802899292 expires 2024-09-11T22:47:16.118298+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3004982929 expires 2024-09-11T22:48:31.958130+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/363502482 expires 2024-09-11T22:48:31.958130+0000 2024-09-10T22:59:05.587 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1673435000 expires 2024-09-11T22:48:31.958130+0000 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 msgr2=0x7f3c540b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 0x7f3c540b77f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000abe0 tx=0x7f3c6000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 msgr2=0x7f3c7019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c02a910 tx=0x7f3c5c003c20 comp rx=0 tx=0).stop 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 shutdown_connections 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3c540b5330 0x7f3c540b77f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c70113b20 0x7f3c701a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70075e60 0x7f3c7019fcc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 --2- 172.21.15.67:0/2033246301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70075540 0x7f3c7019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.583+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 >> 172.21.15.67:0/2033246301 conn(0x7f3c700fe830 msgr2=0x7f3c700ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.584+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 shutdown_connections 2024-09-10T22:59:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:05.584+0000 7f3c76200640 1 -- 172.21.15.67:0/2033246301 wait complete. 2024-09-10T22:59:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:05 smithi092 ceph-mon[34223]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:06.525 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T22:59:06.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:06.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:06 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2033246301' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:59:06.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:06 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2033246301' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:59:06.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:06 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2033246301' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:59:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:07.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:07 smithi067 ceph-mon[30939]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:07.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 -- 172.21.15.67:0/1288349740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 msgr2=0x7f0bdc10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 --2- 172.21.15.67:0/1288349740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc10ae10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 -- 172.21.15.67:0/1288349740 shutdown_connections 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 --2- 172.21.15.67:0/1288349740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc10b350 0x7f0bdc111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 --2- 172.21.15.67:0/1288349740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc10ae10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 --2- 172.21.15.67:0/1288349740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 -- 172.21.15.67:0/1288349740 >> 172.21.15.67:0/1288349740 conn(0x7f0bdc1009f0 msgr2=0x7f0bdc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 -- 172.21.15.67:0/1288349740 shutdown_connections 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.505+0000 7f0be0a00640 1 -- 172.21.15.67:0/1288349740 wait complete. 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.506+0000 7f0be0a00640 1 Processor -- start 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.506+0000 7f0be0a00640 1 -- start start 2024-09-10T22:59:07.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 0x7f0bdc1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114f50 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114dd0 con 0x7f0bdc104fc0 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0be0a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1150d0 con 0x7f0bdc10b350 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57080/0 (socket says 172.21.15.67:57080) 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 -- 172.21.15.67:0/2321276814 learned_addr learned my addr 172.21.15.67:0/2321276814 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd9e00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bda800640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 0x7f0bdc1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 -- 172.21.15.67:0/2321276814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 msgr2=0x7f0bdc1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 0x7f0bdc1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 -- 172.21.15.67:0/2321276814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 msgr2=0x7f0bdc19b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc19b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd1400640 1 -- 172.21.15.67:0/2321276814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1a15b0 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.507+0000 7f0bd9e00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0bda800640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 0x7f0bdc1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0bd1400640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0bc402a930 tx=0x7f0bc4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc403f070 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc403bdc0 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4035030 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a1840 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.508+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a7f60 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.509+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc105420 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.511+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f0bc4031500 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.513+0000 7f0bd3400640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 0x7f0bc00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.513+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f0bc4106e80 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.513+0000 7f0bd9e00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 0x7f0bc00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.514+0000 7f0bd9e00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 0x7f0bc00b7870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009930 tx=0x7f0bcc008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.517+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0bc40cd830 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:07 smithi145 ceph-mon[34397]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:07.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.751+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bdc10cba0 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.752+0000 7f0bd3400640 1 -- 172.21.15.67:0/2321276814 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f0bc40cd130 con 0x7f0bdc10a9d0 2024-09-10T22:59:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:07.754 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:59:07.754 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:59:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 msgr2=0x7f0bc00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 0x7f0bc00b7870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009930 tx=0x7f0bcc008040 comp rx=0 tx=0).stop 2024-09-10T22:59:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 msgr2=0x7f0bdc19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0bc402a930 tx=0x7f0bc4003d60 comp rx=0 tx=0).stop 2024-09-10T22:59:07.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 shutdown_connections 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0bc00b53b0 0x7f0bc00b7870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc10b350 0x7f0bdc1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc10a9d0 0x7f0bdc19ba40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 --2- 172.21.15.67:0/2321276814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bdc104fc0 0x7f0bdc19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 >> 172.21.15.67:0/2321276814 conn(0x7f0bdc1009f0 msgr2=0x7f0bdc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.754+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 shutdown_connections 2024-09-10T22:59:07.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:07.755+0000 7f0be0a00640 1 -- 172.21.15.67:0/2321276814 wait complete. 2024-09-10T22:59:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:07 smithi092 ceph-mon[34223]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:08.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:08 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2321276814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:08.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:08 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2321276814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:08.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:08.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:08 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2321276814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:09.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:09 smithi067 ceph-mon[30939]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:09.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:09 smithi145 ceph-mon[34397]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 -- 172.21.15.67:0/1317959365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 msgr2=0x7fa8841117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1317959365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841117d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009ca0 tx=0x7fa87c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 -- 172.21.15.67:0/1317959365 shutdown_connections 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1317959365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841117d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1317959365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075f40 0x7fa88410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1317959365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075540 0x7fa884075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 -- 172.21.15.67:0/1317959365 >> 172.21.15.67:0/1317959365 conn(0x7fa8840fe830 msgr2=0x7fa884100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 -- 172.21.15.67:0/1317959365 shutdown_connections 2024-09-10T22:59:09.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.748+0000 7fa88ae00640 1 -- 172.21.15.67:0/1317959365 wait complete. 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.749+0000 7fa88ae00640 1 Processor -- start 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 -- start start 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075540 0x7fa88419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa884114e60 con 0x7fa88410f3e0 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa884114ce0 con 0x7fa884075f40 2024-09-10T22:59:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa88ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa884114fe0 con 0x7fa884075540 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.750+0000 7fa888e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:34168/0 (socket says 172.21.15.67:34168) 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 -- 172.21.15.67:0/1993141511 learned_addr learned my addr 172.21.15.67:0/1993141511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa883e00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075540 0x7fa88419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 -- 172.21.15.67:0/1993141511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075540 msgr2=0x7fa88419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075540 0x7fa88419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 -- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 msgr2=0x7fa8841a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 -- 172.21.15.67:0/1993141511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8841a1480 con 0x7fa884075f40 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa888e00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.751+0000 7fa87b400640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d950 tx=0x7fa86c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.752+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c017030 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.752+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c004020 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.752+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0055d0 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.752+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8841a1770 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.752+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8841a7e30 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.753+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa884075920 con 0x7fa884075f40 2024-09-10T22:59:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.756+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fa86c009a30 con 0x7fa884075f40 2024-09-10T22:59:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.757+0000 7fa881600640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 0x7fa8680b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.757+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa86c0e6090 con 0x7fa884075f40 2024-09-10T22:59:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.757+0000 7fa883e00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 0x7fa8680b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.758+0000 7fa883e00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 0x7fa8680b7870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bbe0 tx=0x7fa87400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.759+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa86c0acac0 con 0x7fa884075f40 2024-09-10T22:59:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:09 smithi092 ceph-mon[34223]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:09.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.994+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa88402e5c0 con 0x7fa884075f40 2024-09-10T22:59:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.995+0000 7fa881600640 1 -- 172.21.15.67:0/1993141511 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7fa86c0ac3c0 con 0x7fa884075f40 2024-09-10T22:59:09.997 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:09.998 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"btime":"2024-09-10T22:57:15:423621+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:57:14.572524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T22:59:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 msgr2=0x7fa8680b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 0x7fa8680b7870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bbe0 tx=0x7fa87400b390 comp rx=0 tx=0).stop 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 msgr2=0x7fa88419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d950 tx=0x7fa86c00de10 comp rx=0 tx=0).stop 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 shutdown_connections 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa8680b53b0 0x7fa8680b7870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa88410f3e0 0x7fa8841a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa884075f40 0x7fa88419b910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:10.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 --2- 172.21.15.67:0/1993141511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa884075540 0x7fa88419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:10.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 >> 172.21.15.67:0/1993141511 conn(0x7fa8840fe830 msgr2=0x7fa8840ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:10.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 shutdown_connections 2024-09-10T22:59:10.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:09.999+0000 7fa88ae00640 1 -- 172.21.15.67:0/1993141511 wait complete. 2024-09-10T22:59:10.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:10 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1993141511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:10.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:10 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1993141511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:10.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:10 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1993141511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:10.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:59:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:11.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:11 smithi067 ceph-mon[30939]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:11.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:11 smithi145 ceph-mon[34397]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:11 smithi092 ceph-mon[34223]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:11.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 -- 172.21.15.67:0/2236709561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc10f3e0 msgr2=0x7f98fc1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 --2- 172.21.15.67:0/2236709561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc10f3e0 0x7f98fc1117d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009ca0 tx=0x7f98f802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 -- 172.21.15.67:0/2236709561 shutdown_connections 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 --2- 172.21.15.67:0/2236709561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc10f3e0 0x7f98fc1117d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 --2- 172.21.15.67:0/2236709561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98fc075f40 0x7f98fc10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 --2- 172.21.15.67:0/2236709561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075540 0x7f98fc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 -- 172.21.15.67:0/2236709561 >> 172.21.15.67:0/2236709561 conn(0x7f98fc0fe830 msgr2=0x7f98fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.907+0000 7f9904800640 1 -- 172.21.15.67:0/2236709561 shutdown_connections 2024-09-10T22:59:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.908+0000 7f9904800640 1 -- 172.21.15.67:0/2236709561 wait complete. 2024-09-10T22:59:11.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.908+0000 7f9904800640 1 Processor -- start 2024-09-10T22:59:11.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 -- start start 2024-09-10T22:59:11.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:11.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075f40 0x7f98fc19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98fc10f3e0 0x7f98fc1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc114e60 con 0x7f98fc075540 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc114ce0 con 0x7f98fc075f40 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9904800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc114fe0 con 0x7f98fc10f3e0 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9901e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9901e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38942/0 (socket says 172.21.15.67:38942) 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.909+0000 7f9901e00640 1 -- 172.21.15.67:0/1096832362 learned_addr learned my addr 172.21.15.67:0/1096832362 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901400640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075f40 0x7f98fc19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 -- 172.21.15.67:0/1096832362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98fc10f3e0 msgr2=0x7f98fc1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98fc10f3e0 0x7f98fc1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 -- 172.21.15.67:0/1096832362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075f40 msgr2=0x7f98fc19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075f40 0x7f98fc19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 -- 172.21.15.67:0/1096832362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1a13e0 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9901e00640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98f000d950 tx=0x7f98f000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0017070 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc1a16d0 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.910+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98f000de80 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.911+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000f620 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.911+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1a7d90 con 0x7f98fc075540 2024-09-10T22:59:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.912+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc075920 con 0x7f98fc075540 2024-09-10T22:59:11.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.914+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f98f0005030 con 0x7f98fc075540 2024-09-10T22:59:11.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.915+0000 7f98f6a00640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 0x7f98e40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:11.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.916+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f98f00e5d60 con 0x7f98fc075540 2024-09-10T22:59:11.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.916+0000 7f9901400640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 0x7f98e40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:11.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.916+0000 7f9901400640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 0x7f98e40b77f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98fc1010e0 tx=0x7f98ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:11.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:11.917+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f98f00ac710 con 0x7f98fc075540 2024-09-10T22:59:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.114+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98fc02e5c0 con 0x7f98fc075540 2024-09-10T22:59:12.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.115+0000 7f98f6a00640 1 -- 172.21.15.67:0/1096832362 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25221 (secure 0 0 0) 0x7f98f00ac010 con 0x7f98fc075540 2024-09-10T22:59:12.117 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:12.118 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:58:34.175811+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 msgr2=0x7f98e40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 0x7f98e40b77f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98fc1010e0 tx=0x7f98ec00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 msgr2=0x7f98fc19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98f000d950 tx=0x7f98f000dd50 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 shutdown_connections 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f98e40b5330 0x7f98e40b77f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98fc10f3e0 0x7f98fc1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98fc075f40 0x7f98fc19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 --2- 172.21.15.67:0/1096832362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98fc075540 0x7f98fc19b330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 >> 172.21.15.67:0/1096832362 conn(0x7f98fc0fe830 msgr2=0x7f98fc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 shutdown_connections 2024-09-10T22:59:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:12.118+0000 7f9904800640 1 -- 172.21.15.67:0/1096832362 wait complete. 2024-09-10T22:59:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:12 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1096832362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:12.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:12 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1096832362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:12.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:12 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1096832362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:13.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:56:19.886591+0000', 'modified': '2024-09-10T22:57:14.572524+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34730, 'mds_1': 34691, 'mds_2': 34631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24664': {'gid': 24664, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/2757139462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 2757139462}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 2757139462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/1285378679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 1285378679}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 1285378679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24691': {'gid': 24691, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6839/4174586003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 4174586003}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 4174586003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34631': {'gid': 34631, 'name': 'k', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.92:6833/307002386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 307002386}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 307002386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34691': {'gid': 34691, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.92:6837/523163273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 523163273}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 523163273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34730': {'gid': 34730, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/1949986898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1949986898}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1949986898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34730, 'qdb_cluster': [34730, 34691, 34631]} 2024-09-10T22:59:13.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T22:59:13.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:59:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:13.618 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:13 smithi067 ceph-mon[30939]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:13.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:13 smithi145 ceph-mon[34397]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:13.673 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:13 smithi092 ceph-mon[34223]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:14.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.018+0000 7f4083200640 1 -- 172.21.15.67:0/3583764319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 msgr2=0x7f407c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.018+0000 7f4083200640 1 --2- 172.21.15.67:0/3583764319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c103180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f406c009900 tx=0x7f406c02a410 comp rx=0 tx=0).stop 2024-09-10T22:59:14.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 -- 172.21.15.67:0/3583764319 shutdown_connections 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 --2- 172.21.15.67:0/3583764319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 --2- 172.21.15.67:0/3583764319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 0x7f407c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 --2- 172.21.15.67:0/3583764319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c103180 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 -- 172.21.15.67:0/3583764319 >> 172.21.15.67:0/3583764319 conn(0x7f407c0fe810 msgr2=0x7f407c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 -- 172.21.15.67:0/3583764319 shutdown_connections 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.019+0000 7f4083200640 1 -- 172.21.15.67:0/3583764319 wait complete. 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.020+0000 7f4083200640 1 Processor -- start 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.020+0000 7f4083200640 1 -- start start 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 0x7f407c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112ca0 con 0x7f407c102da0 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112b20 con 0x7f407c1087b0 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4083200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112e20 con 0x7f407c109130 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38956/0 (socket says 172.21.15.67:38956) 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 -- 172.21.15.67:0/1422155741 learned_addr learned my addr 172.21.15.67:0/1422155741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4073400640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 0x7f407c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4081200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 -- 172.21.15.67:0/1422155741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 msgr2=0x7f407c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 -- 172.21.15.67:0/1422155741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 msgr2=0x7f407c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 0x7f407c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f407be00640 1 -- 172.21.15.67:0/1422155741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c19f260 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.021+0000 7f4081200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f407be00640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f406c009230 tx=0x7f406c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f4079600640 1 -- 172.21.15.67:0/1422155741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c03f030 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f4079600640 1 -- 172.21.15.67:0/1422155741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f406c0043d0 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c19f4f0 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f4079600640 1 -- 172.21.15.67:0/1422155741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c035070 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.022+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c1a5c10 con 0x7f407c102da0 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.025+0000 7f4085e08740 1 Processor -- start 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.025+0000 7f4085e08740 1 -- start start 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da74984a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da749909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55da748f7d50 con 0x55da74984f70 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da748e7180 con 0x55da74984650 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4085e08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55da748eaf90 con 0x55da74990f00 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4079600640 1 -- 172.21.15.67:0/1422155741 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f406c003bf0 con 0x7f407c102da0 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f407be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da749909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4073400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4073400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40334/0 (socket says 172.21.15.67:40334) 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4073400640 1 -- 172.21.15.67:0/3098687633 learned_addr learned my addr 172.21.15.67:0/3098687633 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.026+0000 7f4081200640 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da74984a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 -- 172.21.15.67:0/3098687633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 msgr2=0x55da74984a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da74984a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 -- 172.21.15.67:0/3098687633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 msgr2=0x55da749909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da749909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 -- 172.21.15.67:0/3098687633 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da748b1a70 con 0x55da74990f00 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4081200640 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da74984a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4073400640 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4064009c90 tx=0x7f4064011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebbbe9c94d1ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4071600640 1 -- 172.21.15.67:0/3098687633 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406401e070 con 0x55da74990f00 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4071600640 1 -- 172.21.15.67:0/3098687633 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4064019ac0 con 0x55da74990f00 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.027+0000 7f4071600640 1 -- 172.21.15.67:0/3098687633 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406400e880 con 0x55da74990f00 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4079600640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 0x7f40600b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4079600640 1 -- 172.21.15.67:0/1422155741 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f406c1067b0 con 0x7f407c102da0 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 -- 172.21.15.67:0/3098687633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 msgr2=0x55da749932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4064009c90 tx=0x7f4064011810 comp rx=0 tx=0).stop 2024-09-10T22:59:14.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4081200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 0x7f40600b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 -- 172.21.15.67:0/3098687633 shutdown_connections 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da749932f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da749909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 --2- 172.21.15.67:0/3098687633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da74984a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 -- 172.21.15.67:0/3098687633 >> 172.21.15.67:0/3098687633 conn(0x55da74982430 msgr2=0x55da74983620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 -- 172.21.15.67:0/3098687633 shutdown_connections 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.028+0000 7f4085e08740 1 -- 172.21.15.67:0/3098687633 wait complete. 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.029+0000 7f4081200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 0x7f40600b7870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4074005fd0 tx=0x7f4074009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.029+0000 7f4085e08740 1 -- start start 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.029+0000 7f4085e08740 1 Processor -- start 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da749909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da7498e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55da747485f0 con 0x55da74984f70 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da74882460 con 0x55da74984650 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55da74993c20 con 0x55da74990f00 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4085e08740 10 client.0 _ll_register_callbacks cb 0x55da74892210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f407be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da749909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38974/0 (socket says 172.21.15.67:38974) 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 -- 172.21.15.67:0/6595639 learned_addr learned my addr 172.21.15.67:0/6595639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:14.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4081200640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da7498e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 -- 172.21.15.67:0/6595639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 msgr2=0x55da7498e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da7498e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 -- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 msgr2=0x55da749909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.030+0000 7f4073400640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da749909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f4073400640 1 -- 172.21.15.67:0/6595639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da74645540 con 0x55da74984f70 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f407be00640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da749909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f4073400640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4064005750 tx=0x7f4064005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3b9900de882810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f406a000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406401e070 con 0x55da74984f70 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.031+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da74884a70 con 0x55da74984f70 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.032+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f4064004230 con 0x55da74984f70 2024-09-10T22:59:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.032+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4064004470 con 0x55da74984f70 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.032+0000 7f4085e08740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.032+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da7489f6f0 con 0x55da74984f70 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.032+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55da74884a70 con 0x55da74984f70 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4064003a70 con 0x55da74984f70 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 10 client.15462 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 20 client.15462 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da749a19c0 0x55da749a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 4 client.15462 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da748f9ca0 con 0x55da749a19c0 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.033+0000 7f407be00640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da749a19c0 0x55da749a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.034+0000 7f407be00640 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da749a19c0 0x55da749a2e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a749c005189b66f server_cookie=cf6d65a8f0a949c in_seq=0 out_seq=0 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.034+0000 7f406a000640 10 client.15462 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f406a000640 1 -- 172.21.15.67:0/6595639 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da748f9ca0 con 0x55da749a19c0 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f406a000640 10 client.15462 handle_command_reply: tid=0 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 client.15462 shutdown 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 msgr2=0x55da7498de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4064005750 tx=0x7f4064005780 comp rx=0 tx=0).stop 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 shutdown_connections 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da749a19c0 0x55da749a2e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da7498e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984f70 0x55da7498de50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 --2- 172.21.15.67:0/6595639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984650 0x55da749909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 >> 172.21.15.67:0/6595639 conn(0x55da74982430 msgr2=0x55da7497d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f406a000640 0 client.15462 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.035+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 shutdown_connections 2024-09-10T22:59:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.036+0000 7f4085e08740 1 -- 172.21.15.67:0/6595639 wait complete. 2024-09-10T22:59:14.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.036+0000 7f4085e08740 20 client.15462 trim_cache size 0 max 16384 2024-09-10T22:59:14.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 Processor -- start 2024-09-10T22:59:14.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 -- start start 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da74989100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da74989640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55da749a1170 con 0x55da74984650 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da749a1840 con 0x55da74984f70 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f4085e08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55da749a1f10 con 0x55da74990f00 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f407be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f407be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38986/0 (socket says 172.21.15.67:38986) 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.055+0000 7f407be00640 1 -- 172.21.15.67:0/854795550 learned_addr learned my addr 172.21.15.67:0/854795550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4073400640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da74989100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4081200640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da74989640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 -- 172.21.15.67:0/854795550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 msgr2=0x55da74989640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da74989640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 -- 172.21.15.67:0/854795550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 msgr2=0x55da74989100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da74989100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 -- 172.21.15.67:0/854795550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da74984da0 con 0x55da74984650 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4081200640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da74989640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4073400640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da74989100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f407be00640 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f406c0cc9f0 tx=0x7f406c103c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e09817bec03387d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4070c00640 1 -- 172.21.15.67:0/854795550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c049040 con 0x55da74984650 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4070c00640 1 -- 172.21.15.67:0/854795550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f406c0cc570 con 0x55da74984650 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.056+0000 7f4070c00640 1 -- 172.21.15.67:0/854795550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c0cca60 con 0x55da74984650 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 -- 172.21.15.67:0/854795550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 msgr2=0x55da749a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f406c0cc9f0 tx=0x7f406c103c70 comp rx=0 tx=0).stop 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 -- 172.21.15.67:0/854795550 shutdown_connections 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74990f00 0x55da74989640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da74989100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 --2- 172.21.15.67:0/854795550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74984650 0x55da749a2e30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 -- 172.21.15.67:0/854795550 >> 172.21.15.67:0/854795550 conn(0x55da74982430 msgr2=0x55da749a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 -- 172.21.15.67:0/854795550 shutdown_connections 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.057+0000 7f4085e08740 1 -- 172.21.15.67:0/854795550 wait complete. 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 -- start start 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 Processor -- start 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74984650 0x55da7497cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da7497d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55da749a2cb0 con 0x55da74990f00 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55da749a25e0 con 0x55da74984f70 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55da749a3380 con 0x55da74984650 2024-09-10T22:59:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.058+0000 7f4085e08740 10 client.0 _ll_register_callbacks cb 0x7f4064009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38992/0 (socket says 172.21.15.67:38992) 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 -- 172.21.15.67:0/3283743750 learned_addr learned my addr 172.21.15.67:0/3283743750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4073400640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da7497d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f407be00640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74984650 0x55da7497cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 -- 172.21.15.67:0/3283743750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74984650 msgr2=0x55da7497cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74984650 0x55da7497cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 -- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 msgr2=0x55da7497d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da7497d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 -- 172.21.15.67:0/3283743750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da749abb10 con 0x55da74990f00 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f4081200640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4074010c20 tx=0x7f4074014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1637ad5069c8a336 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.059+0000 7f406a000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:14.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074027600 con 0x55da74990f00 2024-09-10T22:59:14.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f40740277a0 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f406c0cc570 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4074032050 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f4085e08740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da749a6da0 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.060+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f406c0cc570 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4074025ad0 con 0x55da74990f00 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 10 client.15468 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 20 client.15468 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da74a12c50 0x55da74a11960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 4 client.15468 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "status"}) -- 0x55da748f5750 con 0x55da74a12c50 2024-09-10T22:59:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.061+0000 7f407be00640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da74a12c50 0x55da74a11960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.062+0000 7f407be00640 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da74a12c50 0x55da74a11960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151887e4a6a18cd7 server_cookie=3a19924de142f371 in_seq=0 out_seq=0 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.062+0000 7f406a000640 10 client.15468 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.062+0000 7f406a000640 1 -- 172.21.15.67:0/3283743750 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55da748f5750 con 0x55da74a12c50 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.062+0000 7f406a000640 10 client.15468 handle_command_reply: tid=0 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.062+0000 7f4085e08740 1 client.15468 shutdown 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 msgr2=0x55da749ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4074010c20 tx=0x7f4074014040 comp rx=0 tx=0).stop 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 shutdown_connections 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x55da74a12c50 0x55da74a11960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da74990f00 0x55da749ab2e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55da74984f70 0x55da7497d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 --2- 172.21.15.67:0/3283743750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55da74984650 0x55da7497cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f406a000640 0 client.15468 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:14.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 >> 172.21.15.67:0/3283743750 conn(0x55da74982430 msgr2=0x55da74989100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 shutdown_connections 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 1 -- 172.21.15.67:0/3283743750 wait complete. 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.063+0000 7f4085e08740 20 client.15468 trim_cache size 0 max 16384 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34631, 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:14.068 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 123.65065638199999, 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 223.05392008499999 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 msgr2=0x7f40600b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 0x7f40600b7870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4074005fd0 tx=0x7f4074009500 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 msgr2=0x7f407c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f406c009230 tx=0x7f406c0319a0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 shutdown_connections 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f40600b53b0 0x7f40600b7870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f407c109130 0x7f407c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f407c1087b0 0x7f407c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 --2- 172.21.15.67:0/1422155741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f407c102da0 0x7f407c1991b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 >> 172.21.15.67:0/1422155741 conn(0x7f407c0fe810 msgr2=0x7f407c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 shutdown_connections 2024-09-10T22:59:14.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:14.064+0000 7f4083200640 1 -- 172.21.15.67:0/1422155741 wait complete. 2024-09-10T22:59:14.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:59:15.277 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:15.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:15 smithi067 ceph-mon[30939]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:15 smithi145 ceph-mon[34397]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:15 smithi092 ceph-mon[34223]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 -- 172.21.15.67:0/932457528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 msgr2=0x7fcb7c115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 --2- 172.21.15.67:0/932457528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c115f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009ca0 tx=0x7fcb7402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 -- 172.21.15.67:0/932457528 shutdown_connections 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 --2- 172.21.15.67:0/932457528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c115f10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 --2- 172.21.15.67:0/932457528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 0x7fcb7c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 --2- 172.21.15.67:0/932457528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 -- 172.21.15.67:0/932457528 >> 172.21.15.67:0/932457528 conn(0x7fcb7c0fe830 msgr2=0x7fcb7c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 -- 172.21.15.67:0/932457528 shutdown_connections 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.895+0000 7fcb82800640 1 -- 172.21.15.67:0/932457528 wait complete. 2024-09-10T22:59:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.896+0000 7fcb82800640 1 Processor -- start 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 -- start start 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 0x7fcb7c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c119290 con 0x7fcb7c113b20 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c119110 con 0x7fcb7c075540 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.897+0000 7fcb82800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c119410 con 0x7fcb7c075e60 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39016/0 (socket says 172.21.15.67:39016) 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7be00640 1 -- 172.21.15.67:0/3208391756 learned_addr learned my addr 172.21.15.67:0/3208391756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 0x7fcb7c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 -- 172.21.15.67:0/3208391756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 msgr2=0x7fcb7c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 0x7fcb7c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 -- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 msgr2=0x7fcb7c1a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 -- 172.21.15.67:0/3208391756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c1a5850 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7be00640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.898+0000 7fcb7b400640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c19f7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e9f0 tx=0x7fcb6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.899+0000 7fcb73e00640 1 -- 172.21.15.67:0/3208391756 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00ccc0 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.899+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c1a5b40 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.899+0000 7fcb73e00640 1 -- 172.21.15.67:0/3208391756 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb6c0108c0 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.899+0000 7fcb73e00640 1 -- 172.21.15.67:0/3208391756 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0187c0 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.899+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1ac200 con 0x7fcb7c075540 2024-09-10T22:59:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 Processor -- start 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 -- start start 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560f3374fd50 con 0x560f337d8240 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f3373f180 con 0x560f337d8b60 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.902+0000 7fcb8526f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f33742f90 con 0x560f337e4af0 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39024/0 (socket says 172.21.15.67:39024) 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 -- 172.21.15.67:0/2710713865 learned_addr learned my addr 172.21.15.67:0/2710713865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb73e00640 1 -- 172.21.15.67:0/3208391756 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fcb6c010430 con 0x7fcb7c075540 2024-09-10T22:59:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 -- 172.21.15.67:0/2710713865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 msgr2=0x560f337e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337e6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 -- 172.21.15.67:0/2710713865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 msgr2=0x560f337e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7be00640 1 -- 172.21.15.67:0/2710713865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f33709a70 con 0x560f337d8240 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.903+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.904+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005f70 tx=0x7fcb74005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361a2730059d9eab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.904+0000 7fcb70c00640 1 -- 172.21.15.67:0/2710713865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74031c40 con 0x560f337d8240 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.904+0000 7fcb70c00640 1 -- 172.21.15.67:0/2710713865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb74036070 con 0x560f337d8240 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.904+0000 7fcb70c00640 1 -- 172.21.15.67:0/2710713865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740046c0 con 0x560f337d8240 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 -- 172.21.15.67:0/2710713865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 msgr2=0x560f337d8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005f70 tx=0x7fcb74005fa0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb73e00640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 0x7fcb600b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb73e00640 1 -- 172.21.15.67:0/3208391756 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fcb6c0e6420 con 0x7fcb7c075540 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 -- 172.21.15.67:0/2710713865 shutdown_connections 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2710713865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337d8620 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 -- 172.21.15.67:0/2710713865 >> 172.21.15.67:0/2710713865 conn(0x560f337d6020 msgr2=0x560f337d7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb7be00640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 0x7fcb600b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 -- 172.21.15.67:0/2710713865 shutdown_connections 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.905+0000 7fcb8526f740 1 -- 172.21.15.67:0/2710713865 wait complete. 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.906+0000 7fcb7be00640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 0x7fcb600b77f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009430 tx=0x7fcb7402a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.906+0000 7fcb8526f740 1 -- start start 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.906+0000 7fcb8526f740 1 Processor -- start 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8240 0x560f337e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337e4af0 0x560f337e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560f335a05f0 con 0x560f337d8b60 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f336da460 con 0x560f337e4af0 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f337e7660 con 0x560f337d8240 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb8526f740 10 client.0 _ll_register_callbacks cb 0x560f336ea210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39038/0 (socket says 172.21.15.67:39038) 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 -- 172.21.15.67:0/2260509042 learned_addr learned my addr 172.21.15.67:0/2260509042 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8240 0x560f337e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337e4af0 0x560f337e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 -- 172.21.15.67:0/2260509042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8240 msgr2=0x560f337e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8240 0x560f337e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 -- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337e4af0 msgr2=0x560f337e2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337e4af0 0x560f337e2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.907+0000 7fcb7aa00640 1 -- 172.21.15.67:0/2260509042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3349d540 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb68012700 tx=0x7fcb68012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7a46b4638a2ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb65600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800fa60 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f336dca70 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fcb6800fc00 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb6800fe00 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb8526f740 10 client.15480 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.908+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f336f76f0 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560f336dca70 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb6801c420 con 0x560f337d8b60 2024-09-10T22:59:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 10 client.15480 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 10 client.15480 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 20 client.15480 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f337f5720 0x560f337f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 4 client.15480 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f33751ca0 con 0x560f337f5720 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.909+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f337f5720 0x560f337f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.910+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f337f5720 0x560f337f6b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d97d80d4173bc69 server_cookie=55e2a1becc48dce8 in_seq=0 out_seq=0 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.910+0000 7fcb65600640 10 client.15480 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb65600640 1 -- 172.21.15.67:0/2260509042 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f33751ca0 con 0x560f337f5720 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb65600640 10 client.15480 handle_command_reply: tid=0 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 client.15480 shutdown 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 msgr2=0x560f337e2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb68012700 tx=0x7fcb68012bc0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 shutdown_connections 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f337f5720 0x560f337f6b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337e4af0 0x560f337e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb65600640 0 client.15480 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8b60 0x560f337e2020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2260509042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8240 0x560f337e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 >> 172.21.15.67:0/2260509042 conn(0x560f337d6020 msgr2=0x560f337dbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 shutdown_connections 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 1 -- 172.21.15.67:0/2260509042 wait complete. 2024-09-10T22:59:15.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.911+0000 7fcb8526f740 20 client.15480 trim_cache size 0 max 16384 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.930+0000 7fcb8526f740 1 Processor -- start 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 -- start start 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560f337f4ed0 con 0x560f337d8240 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f337f55a0 con 0x560f337d8b60 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb8526f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f337f5c70 con 0x560f337e4af0 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39044/0 (socket says 172.21.15.67:39044) 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7b400640 1 -- 172.21.15.67:0/1537509479 learned_addr learned my addr 172.21.15.67:0/1537509479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7be00640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.931+0000 7fcb7b400640 1 -- 172.21.15.67:0/1537509479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 msgr2=0x560f337dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7b400640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7b400640 1 -- 172.21.15.67:0/1537509479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 msgr2=0x560f337dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7b400640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337dcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7b400640 1 -- 172.21.15.67:0/1537509479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f337d7c20 con 0x560f337d8240 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7be00640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb7b400640 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0a99f0 tx=0x7fcb6c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3509180afca9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb67e00640 1 -- 172.21.15.67:0/1537509479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0b5c60 con 0x560f337d8240 2024-09-10T22:59:15.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb67e00640 1 -- 172.21.15.67:0/1537509479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb6c0e6120 con 0x560f337d8240 2024-09-10T22:59:15.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.932+0000 7fcb67e00640 1 -- 172.21.15.67:0/1537509479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0a80d0 con 0x560f337d8240 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- 172.21.15.67:0/1537509479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 msgr2=0x560f337e28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0a99f0 tx=0x7fcb6c00ce60 comp rx=0 tx=0).stop 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- 172.21.15.67:0/1537509479 shutdown_connections 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337e4af0 0x560f337dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8b60 0x560f337dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 --2- 172.21.15.67:0/1537509479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337d8240 0x560f337e28a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- 172.21.15.67:0/1537509479 >> 172.21.15.67:0/1537509479 conn(0x560f337d6020 msgr2=0x560f337d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- 172.21.15.67:0/1537509479 shutdown_connections 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- 172.21.15.67:0/1537509479 wait complete. 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.933+0000 7fcb8526f740 1 -- start start 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 Processor -- start 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8240 0x560f337ff810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8b60 0x560f337f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560f337f6a10 con 0x560f337e4af0 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f337f6340 con 0x560f337d8240 2024-09-10T22:59:15.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f337f70e0 con 0x560f337d8b60 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb8526f740 10 client.0 _ll_register_callbacks cb 0x7fcb68009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39048/0 (socket says 172.21.15.67:39048) 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb7be00640 1 -- 172.21.15.67:0/2127934477 learned_addr learned my addr 172.21.15.67:0/2127934477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.934+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8240 0x560f337ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7aa00640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8b60 0x560f337f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 -- 172.21.15.67:0/2127934477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8b60 msgr2=0x560f337f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8b60 0x560f337f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 -- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8240 msgr2=0x560f337ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8240 0x560f337ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 -- 172.21.15.67:0/2127934477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f337fe470 con 0x560f337e4af0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb7be00640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb74004310 tx=0x7fcb740042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e2fcf3fd0f45bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb65600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.935+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74061060 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c0e6120 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fcb74003b70 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb74031e50 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 10 client.15486 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f33862270 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb6c0e6120 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb74043020 con 0x560f337e4af0 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 10 client.15486 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 20 client.15486 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.936+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f33866af0 0x560f33865800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.937+0000 7fcb8526f740 4 client.15486 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:15.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.937+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "status"}) -- 0x560f3374d750 con 0x560f33866af0 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.937+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f33866af0 0x560f33865800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.937+0000 7fcb7b400640 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f33866af0 0x560f33865800 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db6294ed5cb806ad server_cookie=13a4ffe71ca58393 in_seq=0 out_seq=0 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.937+0000 7fcb65600640 10 client.15486 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb65600640 1 -- 172.21.15.67:0/2127934477 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560f3374d750 con 0x560f33866af0 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb65600640 10 client.15486 handle_command_reply: tid=0 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 client.15486 shutdown 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 msgr2=0x560f337f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb74004310 tx=0x7fcb740042a0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 shutdown_connections 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x560f33866af0 0x560f33865800 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560f337e4af0 0x560f337f94f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb65600640 0 client.15486 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f337d8b60 0x560f337f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 --2- 172.21.15.67:0/2127934477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f337d8240 0x560f337ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 >> 172.21.15.67:0/2127934477 conn(0x560f337d6020 msgr2=0x560f337dcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 shutdown_connections 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 1 -- 172.21.15.67:0/2127934477 wait complete. 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.938+0000 7fcb8526f740 20 client.15486 trim_cache size 0 max 16384 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-09-10T22:59:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34691, 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 126.538312981, 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 203.732799664 2024-09-10T22:59:15.945 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 msgr2=0x7fcb600b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 0x7fcb600b77f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009430 tx=0x7fcb7402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 msgr2=0x7fcb7c19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c19f7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e9f0 tx=0x7fcb6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 shutdown_connections 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcb600b5330 0x7fcb600b77f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb7c113b20 0x7fcb7c1a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb7c075e60 0x7fcb7c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 --2- 172.21.15.67:0/3208391756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb7c075540 0x7fcb7c19f7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 >> 172.21.15.67:0/3208391756 conn(0x7fcb7c0fe830 msgr2=0x7fcb7c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 shutdown_connections 2024-09-10T22:59:15.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:15.939+0000 7fcb82800640 1 -- 172.21.15.67:0/3208391756 wait complete. 2024-09-10T22:59:16.827 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:59:17.205 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:17.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:17 smithi067 ceph-mon[30939]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:17 smithi145 ceph-mon[34397]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:17 smithi092 ceph-mon[34223]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 -- 172.21.15.67:0/1238669087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 msgr2=0x7ff2381051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 --2- 172.21.15.67:0/1238669087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff2381051f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009900 tx=0x7ff22c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 -- 172.21.15.67:0/1238669087 shutdown_connections 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 --2- 172.21.15.67:0/1238669087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 0x7ff2381119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 --2- 172.21.15.67:0/1238669087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 0x7ff23810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 --2- 172.21.15.67:0/1238669087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff2381051f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 -- 172.21.15.67:0/1238669087 >> 172.21.15.67:0/1238669087 conn(0x7ff238100840 msgr2=0x7ff238102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 -- 172.21.15.67:0/1238669087 shutdown_connections 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.787+0000 7ff23e800640 1 -- 172.21.15.67:0/1238669087 wait complete. 2024-09-10T22:59:17.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.788+0000 7ff23e800640 1 Processor -- start 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.788+0000 7ff23e800640 1 -- start start 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 0x7ff23819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 0x7ff2381a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff238114d60 con 0x7ff238104e10 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff238114be0 con 0x7ff23810b1a0 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff238114ee0 con 0x7ff23810a820 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39068/0 (socket says 172.21.15.67:39068) 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 -- 172.21.15.67:0/2666278797 learned_addr learned my addr 172.21.15.67:0/2666278797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23ca00640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 0x7ff23819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23de00640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 0x7ff2381a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 -- 172.21.15.67:0/2666278797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 msgr2=0x7ff23819b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 0x7ff23819b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 -- 172.21.15.67:0/2666278797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 msgr2=0x7ff2381a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 0x7ff2381a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.789+0000 7ff23d400640 1 -- 172.21.15.67:0/2666278797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2381a1280 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff23d400640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02a910 tx=0x7ff22c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff236000640 1 -- 172.21.15.67:0/2666278797 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c03f070 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff236000640 1 -- 172.21.15.67:0/2666278797 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff22c003e40 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2381a1510 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff236000640 1 -- 172.21.15.67:0/2666278797 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c035030 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.790+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a7c30 con 0x7ff238104e10 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.793+0000 7ff242896740 1 Processor -- start 2024-09-10T22:59:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.793+0000 7ff242896740 1 -- start start 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26caef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff236000640 1 -- 172.21.15.67:0/2666278797 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7ff22c031400 con 0x7ff238104e10 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26caf20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c26ca5ad50 con 0x55c26cae3400 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c26ca4a180 con 0x55c26caefcb0 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff242896740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c26ca4df90 con 0x55c26cae3d20 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39076/0 (socket says 172.21.15.67:39076) 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff23de00640 1 -- 172.21.15.67:0/3274221988 learned_addr learned my addr 172.21.15.67:0/3274221988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff23d400640 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26caef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.794+0000 7ff23ca00640 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26caf20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 -- 172.21.15.67:0/3274221988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 msgr2=0x55c26caef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26caef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 -- 172.21.15.67:0/3274221988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 msgr2=0x55c26caf20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26caf20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 -- 172.21.15.67:0/3274221988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26ca14a70 con 0x55c26cae3400 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff23de00640 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff230009e90 tx=0x7ff230010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc23a3963f066d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff22a600640 1 -- 172.21.15.67:0/3274221988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230016070 con 0x55c26cae3400 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff22a600640 1 -- 172.21.15.67:0/3274221988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff230004d20 con 0x55c26cae3400 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff22a600640 1 -- 172.21.15.67:0/3274221988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230012650 con 0x55c26cae3400 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.795+0000 7ff236000640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 0x7ff2200b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff236000640 1 -- 172.21.15.67:0/2666278797 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff22c106c30 con 0x7ff238104e10 2024-09-10T22:59:17.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff23d400640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 0x7ff2200b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 -- 172.21.15.67:0/3274221988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 msgr2=0x55c26cae37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff230009e90 tx=0x7ff230010740 comp rx=0 tx=0).stop 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 -- 172.21.15.67:0/3274221988 shutdown_connections 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26caf20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26caef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 --2- 172.21.15.67:0/3274221988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26cae37e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 -- 172.21.15.67:0/3274221988 >> 172.21.15.67:0/3274221988 conn(0x55c26cae1230 msgr2=0x55c26cae23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 -- 172.21.15.67:0/3274221988 shutdown_connections 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff23d400640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 0x7ff2200b78a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02a9b0 tx=0x7ff22c10a670 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.796+0000 7ff242896740 1 -- 172.21.15.67:0/3274221988 wait complete. 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.797+0000 7ff242896740 1 -- start start 2024-09-10T22:59:17.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.797+0000 7ff242896740 1 Processor -- start 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.797+0000 7ff242896740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3d20 0x55c26caed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26caefcb0 0x55c26caed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c26c8ab5f0 con 0x55c26cae3400 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c26c9e5460 con 0x55c26cae3d20 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c26caf2a00 con 0x55c26caefcb0 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff242896740 10 client.0 _ll_register_callbacks cb 0x55c26c9f5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39082/0 (socket says 172.21.15.67:39082) 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 -- 172.21.15.67:0/1914621177 learned_addr learned my addr 172.21.15.67:0/1914621177 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26caefcb0 0x55c26caed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3d20 0x55c26caed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 -- 172.21.15.67:0/1914621177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26caefcb0 msgr2=0x55c26caed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26caefcb0 0x55c26caed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 -- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3d20 msgr2=0x55c26caed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3d20 0x55c26caed1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.798+0000 7ff23de00640 1 -- 172.21.15.67:0/1914621177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26c7a8540 con 0x55c26cae3400 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff23de00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff230000f80 tx=0x7ff230010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7e897712ecf56f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff21ea00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230016070 con 0x55c26cae3400 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff230029430 con 0x55c26cae3400 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c26c9e7a70 con 0x55c26cae3400 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff230029630 con 0x55c26cae3400 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff242896740 10 client.15498 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:17.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c26ca026f0 con 0x55c26cae3400 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.799+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c26c9e7a70 con 0x55c26cae3400 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff23000f040 con 0x55c26cae3400 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 10 client.15498 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 20 client.15498 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb006a0 0x55c26cb01b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 4 client.15498 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c26ca5cca0 con 0x55c26cb006a0 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.800+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb006a0 0x55c26cb01b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.801+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb006a0 0x55c26cb01b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00ef1f36a250134 server_cookie=2184fad93cce3368 in_seq=0 out_seq=0 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.801+0000 7ff21ea00640 10 client.15498 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:17.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff21ea00640 1 -- 172.21.15.67:0/1914621177 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c26ca5cca0 con 0x55c26cb006a0 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff21ea00640 10 client.15498 handle_command_reply: tid=0 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 client.15498 shutdown 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 msgr2=0x55c26caeccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff230000f80 tx=0x7ff230010f10 comp rx=0 tx=0).stop 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 shutdown_connections 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb006a0 0x55c26cb01b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26caefcb0 0x55c26caed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3d20 0x55c26caed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 --2- 172.21.15.67:0/1914621177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3400 0x55c26caeccb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 >> 172.21.15.67:0/1914621177 conn(0x55c26cae1230 msgr2=0x55c26cae6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff21ea00640 0 client.15498 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 shutdown_connections 2024-09-10T22:59:17.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.802+0000 7ff242896740 1 -- 172.21.15.67:0/1914621177 wait complete. 2024-09-10T22:59:17.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.803+0000 7ff242896740 20 client.15498 trim_cache size 0 max 16384 2024-09-10T22:59:17.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 Processor -- start 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 -- start start 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3400 0x55c26cae9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26cb09df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c26caffe50 con 0x55c26cae3d20 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c26cb00520 con 0x55c26caefcb0 2024-09-10T22:59:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.822+0000 7ff242896740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c26cb00bf0 con 0x55c26cae3400 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39088/0 (socket says 172.21.15.67:39088) 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 -- 172.21.15.67:0/1755155782 learned_addr learned my addr 172.21.15.67:0/1755155782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23de00640 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26cb09df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23de00640 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3400 0x55c26cae9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 -- 172.21.15.67:0/1755155782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3400 msgr2=0x55c26cae9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3400 0x55c26cae9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 -- 172.21.15.67:0/1755155782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 msgr2=0x55c26cb09df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26cb09df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 -- 172.21.15.67:0/1755155782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26cae1060 con 0x55c26cae3d20 2024-09-10T22:59:17.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.823+0000 7ff23ca00640 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff224005d00 tx=0x7ff22400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b865a95cd3105638 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.824+0000 7ff229c00640 1 -- 172.21.15.67:0/1755155782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224004220 con 0x55c26cae3d20 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.824+0000 7ff229c00640 1 -- 172.21.15.67:0/1755155782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2240043c0 con 0x55c26cae3d20 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.824+0000 7ff229c00640 1 -- 172.21.15.67:0/1755155782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224004d30 con 0x55c26cae3d20 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.824+0000 7ff242896740 1 -- 172.21.15.67:0/1755155782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 msgr2=0x55c26caea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.824+0000 7ff242896740 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff224005d00 tx=0x7ff22400b410 comp rx=0 tx=0).stop 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 -- 172.21.15.67:0/1755155782 shutdown_connections 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26caefcb0 0x55c26cb09df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26cae3d20 0x55c26caea420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 --2- 172.21.15.67:0/1755155782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3400 0x55c26cae9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 -- 172.21.15.67:0/1755155782 >> 172.21.15.67:0/1755155782 conn(0x55c26cae1230 msgr2=0x55c26caeb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 -- 172.21.15.67:0/1755155782 shutdown_connections 2024-09-10T22:59:17.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 -- 172.21.15.67:0/1755155782 wait complete. 2024-09-10T22:59:17.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 -- start start 2024-09-10T22:59:17.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.825+0000 7ff242896740 1 Processor -- start 2024-09-10T22:59:17.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3400 0x55c26cb0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26cb0a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c26cb01990 con 0x55c26caefcb0 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c26cb012c0 con 0x55c26cae3400 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c26cb02060 con 0x55c26cae3d20 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff242896740 10 client.0 _ll_register_callbacks cb 0x7ff230009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff23de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39098/0 (socket says 172.21.15.67:39098) 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff23de00640 1 -- 172.21.15.67:0/3445706844 learned_addr learned my addr 172.21.15.67:0/3445706844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.826+0000 7ff23de00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3400 0x55c26cb0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23ca00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26cb0a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 -- 172.21.15.67:0/3445706844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 msgr2=0x55c26cb0a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26cb0a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 -- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3400 msgr2=0x55c26cb0a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3400 0x55c26cb0a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 -- 172.21.15.67:0/3445706844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26cae1cb0 con 0x55c26caefcb0 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff23de00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff230009e90 tx=0x7ff23002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fa8e09891c9166 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff21ea00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.827+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230016040 con 0x55c26caefcb0 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2240043c0 con 0x55c26caefcb0 2024-09-10T22:59:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff23002bc30 con 0x55c26caefcb0 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff230045050 con 0x55c26caefcb0 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff242896740 10 client.15504 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c26cb762b0 con 0x55c26caefcb0 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2240043c0 con 0x55c26caefcb0 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff230029a40 con 0x55c26caefcb0 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.828+0000 7ff242896740 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff242896740 10 client.15504 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff242896740 20 client.15504 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb7ae20 0x55c26cb79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff242896740 4 client.15504 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "status"}) -- 0x55c26ca58750 con 0x55c26cb7ae20 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.829+0000 7ff23ca00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb7ae20 0x55c26cb79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff23ca00640 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb7ae20 0x55c26cb79b30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4f0ac1b55cd002 server_cookie=2d6cf04e60765416 in_seq=0 out_seq=0 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff21ea00640 10 client.15504 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff21ea00640 1 -- 172.21.15.67:0/3445706844 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c26ca58750 con 0x55c26cb7ae20 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff21ea00640 10 client.15504 handle_command_reply: tid=0 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff242896740 1 client.15504 shutdown 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 msgr2=0x55c26cb04e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.830+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff230009e90 tx=0x7ff23002b040 comp rx=0 tx=0).stop 2024-09-10T22:59:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 shutdown_connections 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55c26cb7ae20 0x55c26cb79b30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c26caefcb0 0x55c26cb04e90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c26cae3d20 0x55c26cb0a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 --2- 172.21.15.67:0/3445706844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c26cae3400 0x55c26cb0a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff21ea00640 0 client.15504 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 >> 172.21.15.67:0/3445706844 conn(0x55c26cae1230 msgr2=0x55c26caee2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 shutdown_connections 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 1 -- 172.21.15.67:0/3445706844 wait complete. 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.831+0000 7ff242896740 20 client.15504 trim_cache size 0 max 16384 2024-09-10T22:59:17.835 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34730, 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 177.93359063400001, 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 26, 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 190.286154694 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 msgr2=0x7ff2200b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 0x7ff2200b78a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02a9b0 tx=0x7ff22c10a670 comp rx=0 tx=0).stop 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 msgr2=0x7ff23819b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02a910 tx=0x7ff22c003c20 comp rx=0 tx=0).stop 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 shutdown_connections 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff2200b53e0 0x7ff2200b78a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff23810b1a0 0x7ff2381a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810a820 0x7ff23819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 --2- 172.21.15.67:0/2666278797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff238104e10 0x7ff23819b1d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 >> 172.21.15.67:0/2666278797 conn(0x7ff238100840 msgr2=0x7ff238102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 shutdown_connections 2024-09-10T22:59:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:17.832+0000 7ff23e800640 1 -- 172.21.15.67:0/2666278797 wait complete. 2024-09-10T22:59:18.607 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T22:59:18.620 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T22:59:18.620 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-10T22:59:18.620 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-10T22:59:18.620 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:59:18.621 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:18.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:18.621 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-10T22:59:18.657 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T22:59:18.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:18.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:59:18.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:18.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-09-10T22:59:19.049 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T22:59:19.049 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:59:19.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:59:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:19.570 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:19 smithi067 ceph-mon[30939]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:19 smithi145 ceph-mon[34397]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:19 smithi092 ceph-mon[34223]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 -- 172.21.15.67:0/266704970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8109150 msgr2=0x7fa9d810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/266704970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8109150 0x7fa9d810f950 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009ca0 tx=0x7fa9d002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 -- 172.21.15.67:0/266704970 shutdown_connections 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/266704970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8109150 0x7fa9d810f950 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/266704970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 0x7fa9d8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/266704970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8102dc0 0x7fa9d81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 -- 172.21.15.67:0/266704970 >> 172.21.15.67:0/266704970 conn(0x7fa9d80fe830 msgr2=0x7fa9d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 -- 172.21.15.67:0/266704970 shutdown_connections 2024-09-10T22:59:20.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.009+0000 7fa9dcc00640 1 -- 172.21.15.67:0/266704970 wait complete. 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 Processor -- start 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 -- start start 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 0x7fa9d819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 0x7fa9d81a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8112c80 con 0x7fa9d8102dc0 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8112b00 con 0x7fa9d8109150 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.010+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8112e00 con 0x7fa9d81087d0 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33912/0 (socket says 172.21.15.67:33912) 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 -- 172.21.15.67:0/2072316320 learned_addr learned my addr 172.21.15.67:0/2072316320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9cd400640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 0x7fa9d819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d6800640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 0x7fa9d81a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 -- 172.21.15.67:0/2072316320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 msgr2=0x7fa9d819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 0x7fa9d819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 -- 172.21.15.67:0/2072316320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 msgr2=0x7fa9d81a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 0x7fa9d81a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d5e00640 1 -- 172.21.15.67:0/2072316320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a1450 con 0x7fa9d8102dc0 2024-09-10T22:59:20.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.011+0000 7fa9d6800640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 0x7fa9d81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9d5e00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d950 tx=0x7fa9c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8017070 con 0x7fa9d8102dc0 2024-09-10T22:59:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c8009c30 con 0x7fa9d8102dc0 2024-09-10T22:59:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a1740 con 0x7fa9d8102dc0 2024-09-10T22:59:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800f620 con 0x7fa9d8102dc0 2024-09-10T22:59:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.012+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a7e00 con 0x7fa9d8102dc0 2024-09-10T22:59:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.013+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d81031a0 con 0x7fa9d8102dc0 2024-09-10T22:59:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.015+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fa9c8010460 con 0x7fa9d8102dc0 2024-09-10T22:59:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.018+0000 7fa9cf400640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 0x7fa9bc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:20.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.018+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa9c80e5d80 con 0x7fa9d8102dc0 2024-09-10T22:59:20.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.018+0000 7fa9cd400640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 0x7fa9bc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:20.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.019+0000 7fa9cd400640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 0x7fa9bc0b77f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8104f30 tx=0x7fa9c000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:20.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.020+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa9c80ac730 con 0x7fa9d8102dc0 2024-09-10T22:59:20.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.257+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9d81a8160 con 0x7fa9d8102dc0 2024-09-10T22:59:20.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.258+0000 7fa9cf400640 1 -- 172.21.15.67:0/2072316320 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fa9c8009de0 con 0x7fa9d8102dc0 2024-09-10T22:59:20.260 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 msgr2=0x7fa9bc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 0x7fa9bc0b77f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8104f30 tx=0x7fa9c000b390 comp rx=0 tx=0).stop 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 msgr2=0x7fa9d819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d950 tx=0x7fa9c800de10 comp rx=0 tx=0).stop 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 shutdown_connections 2024-09-10T22:59:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9bc0b5330 0x7fa9bc0b77f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9d8109150 0x7fa9d81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d81087d0 0x7fa9d819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 --2- 172.21.15.67:0/2072316320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9d8102dc0 0x7fa9d819b3a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.260+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 >> 172.21.15.67:0/2072316320 conn(0x7fa9d80fe830 msgr2=0x7fa9d80753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.261+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 shutdown_connections 2024-09-10T22:59:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:20.261+0000 7fa9dcc00640 1 -- 172.21.15.67:0/2072316320 wait complete. 2024-09-10T22:59:20.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2072316320' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:59:20.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2072316320' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:59:20.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2072316320' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:59:21.077 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:59:21.078 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:59:21.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:21.078 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-09-10T22:59:21.117 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:59:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:59:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:59:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-09-10T22:59:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:59:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41945sec preferred_lft 41945sec 2024-09-10T22:59:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link noprefixroute 2024-09-10T22:59:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:59:21.120 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:59:21.120 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:59:21.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:59:21.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:21.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:21.354 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:59:21.410 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-09-10T22:59:21.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:21.413 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-09-10T22:59:21.449 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-09-10T22:59:21.449 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:59:21.449 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:59:21.449 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:59:21.450 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:59:21.450 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:59:21.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:21 smithi067 ceph-mon[30939]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:21.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:59:21.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:21 smithi145 ceph-mon[34397]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:21.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:21.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:21.653 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-10T22:59:21.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:21.689 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-09-10T22:59:21.764 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:59:21.764 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:59:21.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:59:21.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T22:59:21.765 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:59:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:21 smithi092 ceph-mon[34223]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T22:59:21.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:59:21.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:21.945 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T22:59:21.945 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:59:22.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:59:22.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:22.165 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:59:22.165 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-09-10T22:59:22.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-09-10T22:59:22.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:59:22.165 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-09-10T22:59:22.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:59:22.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:22.331 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-10T22:59:22.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:59:22.331 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:22.367 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:59:22.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:59:22.367 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:22.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-10T22:59:22.539 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-10T22:59:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-10T22:59:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/85b4c408ab335fb671d5a6c427ee879da0d5f3c2810ac54e08e3651d6f939a01/merged 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/485979c99d9cd1cd976a54e0849a94a21ab742c1535bfd3fefdaa4ce13d5b15e/merged 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/87080adde8a92b1932f774ea006d4c27dbc818ba6dfff8ad60857a334213e553/merged 2024-09-10T22:59:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3bb0ab0a2a9b9b7981bcc3c7f5f6a51952a3b06915c4c51c9d5e00c29ffa67/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/c306b63195b5a13ec2bde412cee798780c375c1aceb3af80f9ff4b3b76e8d017/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f52f248cff3d648d9c42c1aa1a916c75070ed8cf28516834061492f947436e4/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/411a4a8be1023dc81d20821921a9a2ca0236db2423f190a6e88942bd45a21363/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/ad9e8fd555e64eebea1a315c8ef7e47b2370ec6478a8e456aa0fd860a845750e/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/641cbf72fd4f1b70475507f3ec5d30a4b6fcfc8e187481db8e971de2e423d5be/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f7948be11e2d6942df6aab6a32015b6478914f0b821d683b9c859ced3abaf4ab/merged 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:59:22.579 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:59:22.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:22.580 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-10T22:59:22.656 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:59:22.657 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/8149d884-25b6-438e-a232-f31fd87362b4 --client_fs=cephfs) 2024-09-10T22:59:22.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-09-10T22:59:22.758 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:59:22.796 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-09-10T22:59:22.796 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-09-10T22:59:22.796 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-09-10T22:59:22.796 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-09-10T22:59:22.797 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/85b4c408ab335fb671d5a6c427ee879da0d5f3c2810ac54e08e3651d6f939a01/merged 2024-09-10T22:59:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/485979c99d9cd1cd976a54e0849a94a21ab742c1535bfd3fefdaa4ce13d5b15e/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/87080adde8a92b1932f774ea006d4c27dbc818ba6dfff8ad60857a334213e553/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3bb0ab0a2a9b9b7981bcc3c7f5f6a51952a3b06915c4c51c9d5e00c29ffa67/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/c306b63195b5a13ec2bde412cee798780c375c1aceb3af80f9ff4b3b76e8d017/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/5f52f248cff3d648d9c42c1aa1a916c75070ed8cf28516834061492f947436e4/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/411a4a8be1023dc81d20821921a9a2ca0236db2423f190a6e88942bd45a21363/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/ad9e8fd555e64eebea1a315c8ef7e47b2370ec6478a8e456aa0fd860a845750e/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/641cbf72fd4f1b70475507f3ec5d30a4b6fcfc8e187481db8e971de2e423d5be/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f7948be11e2d6942df6aab6a32015b6478914f0b821d683b9c859ced3abaf4ab/merged 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:59:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:59:22.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:22.800 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-09-10T22:59:22.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-09-10T22:59:22.849+0000 7fb2d4786480 -1 init, newargv = 0x55dc138613a0 newargc=15 2024-09-10T22:59:22.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[89085]: starting ceph client 2024-09-10T22:59:22.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[89085]: starting fuse 2024-09-10T22:59:22.880 INFO:teuthology.orchestra.run.smithi067.stdout:120 2024-09-10T22:59:22.880 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T22:59:22.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-09-10T22:59:22.881 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-09-10T22:59:22.882 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-09-10T22:59:23.045 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.89085.asok 2024-09-10T22:59:23.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:59:23.059 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89085.asok 2024-09-10T22:59:23.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:23.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89085.asok status 2024-09-10T22:59:23.245 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "89085", 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/volumes/qa/sv_0/8149d884-25b6-438e-a232-f31fd87362b4" 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-09-10T22:59:23.246 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 15516, 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 15516 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2366829334 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-10T22:59:23.247 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2366829334 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.15516 172.21.15.67:0/2366829334", 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "172.21.15.67:0/2366829334", 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 5, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 26, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 89, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-09-10T22:59:23.248 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:23.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:23.258 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-09-10T22:59:23.258 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:23.294 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-09-10T22:59:23.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:23.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:59:23.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:23.393 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:59:23.404 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:59:23.405 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:59:23.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T22:59:23.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:23 smithi067 ceph-mon[30939]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:23.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 -- 172.21.15.67:0/3829304259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108500 msgr2=0x7f1d8c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:23.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 --2- 172.21.15.67:0/3829304259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108500 0x7f1d8c108940 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009900 tx=0x7f1d7802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 -- 172.21.15.67:0/3829304259 shutdown_connections 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 --2- 172.21.15.67:0/3829304259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c108e80 0x7f1d8c10f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 --2- 172.21.15.67:0/3829304259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108500 0x7f1d8c108940 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 --2- 172.21.15.67:0/3829304259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c102b00 0x7f1d8c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.559+0000 7f1d92a00640 1 -- 172.21.15.67:0/3829304259 >> 172.21.15.67:0/3829304259 conn(0x7f1d8c0fe580 msgr2=0x7f1d8c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.560+0000 7f1d92a00640 1 -- 172.21.15.67:0/3829304259 shutdown_connections 2024-09-10T22:59:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.560+0000 7f1d92a00640 1 -- 172.21.15.67:0/3829304259 wait complete. 2024-09-10T22:59:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.560+0000 7f1d92a00640 1 Processor -- start 2024-09-10T22:59:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 -- start start 2024-09-10T22:59:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 0x7f1d8c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c108500 0x7f1d8c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c1129f0 con 0x7f1d8c108e80 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112870 con 0x7f1d8c102b00 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c112b70 con 0x7f1d8c108500 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33936/0 (socket says 172.21.15.67:33936) 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d92000640 1 -- 172.21.15.67:0/676076844 learned_addr learned my addr 172.21.15.67:0/676076844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.561+0000 7f1d90c00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c108500 0x7f1d8c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d91600640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 0x7f1d8c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 -- 172.21.15.67:0/676076844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c108500 msgr2=0x7f1d8c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c108500 0x7f1d8c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 -- 172.21.15.67:0/676076844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 msgr2=0x7f1d8c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 0x7f1d8c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 -- 172.21.15.67:0/676076844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c1a73c0 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d91600640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 0x7f1d8c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92000640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d750 tx=0x7f1d8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017070 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1a76b0 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d840054b0 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84020060 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.562+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a7b80 con 0x7f1d8c108e80 2024-09-10T22:59:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.564+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c102ee0 con 0x7f1d8c108e80 2024-09-10T22:59:23.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.566+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f1d84004b30 con 0x7f1d8c108e80 2024-09-10T22:59:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.567+0000 7f1d8a000640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 0x7f1d740b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.567+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f1d840e5c60 con 0x7f1d8c108e80 2024-09-10T22:59:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.567+0000 7f1d91600640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 0x7f1d740b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.568+0000 7f1d91600640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 0x7f1d740b77f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005fd0 tx=0x7f1d80009520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.571+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1d840ac670 con 0x7f1d8c108e80 2024-09-10T22:59:23.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:23 smithi145 ceph-mon[34397]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:23 smithi092 ceph-mon[34223]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T22:59:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:23.804+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f1d8c02e5c0 con 0x7f1d8c108e80 2024-09-10T22:59:24.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.341+0000 7f1d8a000640 1 -- 172.21.15.67:0/676076844 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f1d840abf70 con 0x7f1d8c108e80 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 msgr2=0x7f1d740b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 0x7f1d740b77f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005fd0 tx=0x7f1d80009520 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 msgr2=0x7f1d8c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d750 tx=0x7f1d8400dc10 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 shutdown_connections 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d740b5330 0x7f1d740b77f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d8c108e80 0x7f1d8c075f20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d8c108500 0x7f1d8c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 --2- 172.21.15.67:0/676076844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d8c102b00 0x7f1d8c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 >> 172.21.15.67:0/676076844 conn(0x7f1d8c0fe580 msgr2=0x7f1d8c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 shutdown_connections 2024-09-10T22:59:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.345+0000 7f1d92a00640 1 -- 172.21.15.67:0/676076844 wait complete. 2024-09-10T22:59:24.363 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T22:59:24.374 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:59:24.430 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:59:24.431 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T22:59:24.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-10T22:59:24.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 -- 172.21.15.67:0/2761192088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce7410b0a0 msgr2=0x7fce741118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 --2- 172.21.15.67:0/2761192088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce7410b0a0 0x7fce741118a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009ca0 tx=0x7fce6c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 -- 172.21.15.67:0/2761192088 shutdown_connections 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 --2- 172.21.15.67:0/2761192088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce7410b0a0 0x7fce741118a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 --2- 172.21.15.67:0/2761192088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 --2- 172.21.15.67:0/2761192088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce74104d20 0x7fce74105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 -- 172.21.15.67:0/2761192088 >> 172.21.15.67:0/2761192088 conn(0x7fce74100740 msgr2=0x7fce74102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 -- 172.21.15.67:0/2761192088 shutdown_connections 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.588+0000 7fce73200640 1 -- 172.21.15.67:0/2761192088 wait complete. 2024-09-10T22:59:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 Processor -- start 2024-09-10T22:59:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 -- start start 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7410b0a0 0x7fce741a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fce74114c20 con 0x7fce74104d20 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce74114aa0 con 0x7fce7410b0a0 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.589+0000 7fce73200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce74114da0 con 0x7fce7410a720 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33948/0 (socket says 172.21.15.67:33948) 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 -- 172.21.15.67:0/3484431325 learned_addr learned my addr 172.21.15.67:0/3484431325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce72800640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7410b0a0 0x7fce741a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:24.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce69400640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 -- 172.21.15.67:0/3484431325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 msgr2=0x7fce7419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 -- 172.21.15.67:0/3484431325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7410b0a0 msgr2=0x7fce741a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7410b0a0 0x7fce741a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 -- 172.21.15.67:0/3484431325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741a11e0 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce69400640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.590+0000 7fce71e00640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d950 tx=0x7fce6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.591+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64017030 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.591+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce64004020 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.591+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce741a14d0 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.591+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640055d0 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.591+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a7d70 con 0x7fce74104d20 2024-09-10T22:59:24.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.592+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce74105100 con 0x7fce74104d20 2024-09-10T22:59:24.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.594+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fce64009a30 con 0x7fce74104d20 2024-09-10T22:59:24.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.596+0000 7fce6b400640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 0x7fce580b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:24.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.597+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fce640e5b90 con 0x7fce74104d20 2024-09-10T22:59:24.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.597+0000 7fce69400640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 0x7fce580b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:24.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.597+0000 7fce69400640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 0x7fce580b7870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce74104250 tx=0x7fce5c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:24.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.600+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fce640ac5a0 con 0x7fce74104d20 2024-09-10T22:59:24.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:24 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:59:24.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:24 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:59:24.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:24 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:59:24.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.816+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fce74063590 con 0x7fce580b53b0 2024-09-10T22:59:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.993+0000 7fce6b400640 1 -- 172.21.15.67:0/3484431325 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fce74063590 con 0x7fce580b53b0 2024-09-10T22:59:24.994 INFO:teuthology.orchestra.run.smithi067.stdout:Schedule set for path / 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 msgr2=0x7fce580b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 0x7fce580b7870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce74104250 tx=0x7fce5c005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 msgr2=0x7fce7419b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d950 tx=0x7fce6400de10 comp rx=0 tx=0).stop 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 shutdown_connections 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fce580b53b0 0x7fce580b7870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7410b0a0 0x7fce741a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce7410a720 0x7fce7419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 --2- 172.21.15.67:0/3484431325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce74104d20 0x7fce7419b140 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 >> 172.21.15.67:0/3484431325 conn(0x7fce74100740 msgr2=0x7fce74102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 shutdown_connections 2024-09-10T22:59:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:24.996+0000 7fce73200640 1 -- 172.21.15.67:0/3484431325 wait complete. 2024-09-10T22:59:25.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.169+0000 7f84d7200640 1 -- 172.21.15.67:0/3791366077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 msgr2=0x7f84d0113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.169+0000 7f84d7200640 1 --2- 172.21.15.67:0/3791366077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d0113500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009900 tx=0x7f84b802a410 comp rx=0 tx=0).stop 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 -- 172.21.15.67:0/3791366077 shutdown_connections 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 --2- 172.21.15.67:0/3791366077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 0x7f84d0115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 --2- 172.21.15.67:0/3791366077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d0113500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 --2- 172.21.15.67:0/3791366077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 0x7f84d0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 -- 172.21.15.67:0/3791366077 >> 172.21.15.67:0/3791366077 conn(0x7f84d00fe730 msgr2=0x7f84d0100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 -- 172.21.15.67:0/3791366077 shutdown_connections 2024-09-10T22:59:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 -- 172.21.15.67:0/3791366077 wait complete. 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.170+0000 7f84d7200640 1 Processor -- start 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 -- start start 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 0x7f84d019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 0x7f84d01a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0119180 con 0x7f84d0075e60 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0119000 con 0x7f84d0113a40 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0119300 con 0x7f84d0075540 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33964/0 (socket says 172.21.15.67:33964) 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84c7400640 1 -- 172.21.15.67:0/1893365261 learned_addr learned my addr 172.21.15.67:0/1893365261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84cfe00640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 0x7f84d019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.171+0000 7f84d5200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 0x7f84d01a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 -- 172.21.15.67:0/1893365261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 msgr2=0x7f84d019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 0x7f84d019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 -- 172.21.15.67:0/1893365261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 msgr2=0x7f84d01a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 0x7f84d01a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 -- 172.21.15.67:0/1893365261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d01a58c0 con 0x7f84d0075e60 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84c7400640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009230 tx=0x7f84b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b803f030 con 0x7f84d0075e60 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d01a5b50 con 0x7f84d0075e60 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84b8003a80 con 0x7f84d0075e60 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8035070 con 0x7f84d0075e60 2024-09-10T22:59:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.172+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01ac270 con 0x7f84d0075e60 2024-09-10T22:59:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.173+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0075920 con 0x7f84d0075e60 2024-09-10T22:59:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.176+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f84b803bc30 con 0x7f84d0075e60 2024-09-10T22:59:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.177+0000 7f84cd600640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 0x7f84b40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.177+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f84b8106b80 con 0x7f84d0075e60 2024-09-10T22:59:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.177+0000 7f84cfe00640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 0x7f84b40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.178+0000 7f84cfe00640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 0x7f84b40b77f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fd0 tx=0x7f84c0009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:25.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.181+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f84b80cd530 con 0x7f84d0075e60 2024-09-10T22:59:25.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.399+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f84d0063590 con 0x7f84b40b5330 2024-09-10T22:59:25.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.473+0000 7f84cd600640 1 -- 172.21.15.67:0/1893365261 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f84d0063590 con 0x7f84b40b5330 2024-09-10T22:59:25.474 INFO:teuthology.orchestra.run.smithi067.stdout:Retention added to path / 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 msgr2=0x7f84b40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 0x7f84b40b77f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fd0 tx=0x7f84c0009760 comp rx=0 tx=0).stop 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 msgr2=0x7f84d019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009230 tx=0x7f84b8004270 comp rx=0 tx=0).stop 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 shutdown_connections 2024-09-10T22:59:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84b40b5330 0x7f84b40b77f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84d0113a40 0x7f84d01a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84d0075e60 0x7f84d019fd60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 --2- 172.21.15.67:0/1893365261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84d0075540 0x7f84d019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 >> 172.21.15.67:0/1893365261 conn(0x7f84d00fe730 msgr2=0x7f84d00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 shutdown_connections 2024-09-10T22:59:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.476+0000 7f84d7200640 1 -- 172.21.15.67:0/1893365261 wait complete. 2024-09-10T22:59:25.492 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T22:59:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:25 smithi145 ceph-mon[34397]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T22:59:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:25 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:59:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:25 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:59:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:25 smithi145 ceph-mon[34397]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 -- 172.21.15.67:0/2806061848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10a900 msgr2=0x7f0e9c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 --2- 172.21.15.67:0/2806061848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10a900 0x7f0e9c10ad40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009900 tx=0x7f0e8402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 -- 172.21.15.67:0/2806061848 shutdown_connections 2024-09-10T22:59:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 --2- 172.21.15.67:0/2806061848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10b280 0x7f0e9c111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 --2- 172.21.15.67:0/2806061848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10a900 0x7f0e9c10ad40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 --2- 172.21.15.67:0/2806061848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 0x7f0e9c1052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 -- 172.21.15.67:0/2806061848 >> 172.21.15.67:0/2806061848 conn(0x7f0e9c1008e0 msgr2=0x7f0e9c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 -- 172.21.15.67:0/2806061848 shutdown_connections 2024-09-10T22:59:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.647+0000 7f0ea3800640 1 -- 172.21.15.67:0/2806061848 wait complete. 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.648+0000 7f0ea3800640 1 Processor -- start 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.648+0000 7f0ea3800640 1 -- start start 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 0x7f0e9c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10a900 0x7f0e9c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c114d70 con 0x7f0e9c10b280 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c114bf0 con 0x7f0e9c104f00 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea3800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c114ef0 con 0x7f0e9c10a900 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33992/0 (socket says 172.21.15.67:33992) 2024-09-10T22:59:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 -- 172.21.15.67:0/3725474109 learned_addr learned my addr 172.21.15.67:0/3725474109 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea0e00640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 0x7f0e9c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0e93e00640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10a900 0x7f0e9c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 -- 172.21.15.67:0/3725474109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10a900 msgr2=0x7f0e9c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10a900 0x7f0e9c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 -- 172.21.15.67:0/3725474109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 msgr2=0x7f0e9c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 0x7f0e9c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.649+0000 7f0ea1800640 1 -- 172.21.15.67:0/3725474109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c1a13d0 con 0x7f0e9c10b280 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0ea1800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eaa0 tx=0x7f0e9400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94004d20 con 0x7f0e9c10b280 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e94005020 con 0x7f0e9c10b280 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940116d0 con 0x7f0e9c10b280 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c1a16c0 con 0x7f0e9c10b280 2024-09-10T22:59:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.650+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c1a7d80 con 0x7f0e9c10b280 2024-09-10T22:59:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.651+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c1052e0 con 0x7f0e9c10b280 2024-09-10T22:59:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.654+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f0e9400f040 con 0x7f0e9c10b280 2024-09-10T22:59:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.655+0000 7f0e9a000640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 0x7f0e800b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.655+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f0e940e62f0 con 0x7f0e9c10b280 2024-09-10T22:59:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.655+0000 7f0ea0e00640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 0x7f0e800b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.656+0000 7f0ea0e00640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 0x7f0e800b7870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00abe0 tx=0x7f0e8c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:25.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.657+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0e94016030 con 0x7f0e9c10b280 2024-09-10T22:59:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:25 smithi092 ceph-mon[34223]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T22:59:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:25 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:59:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:25 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:59:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:25 smithi092 ceph-mon[34223]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:25 smithi067 ceph-mon[30939]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T22:59:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:25 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/676076844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:59:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:25 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T22:59:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:25 smithi067 ceph-mon[30939]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.878+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0e9c063590 con 0x7f0e800b53b0 2024-09-10T22:59:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.886+0000 7f0e9a000640 1 -- 172.21.15.67:0/3725474109 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f0e9c063590 con 0x7f0e800b53b0 2024-09-10T22:59:25.887 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T22:59:24", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-10T22:59:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.888+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 msgr2=0x7f0e800b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.888+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 0x7f0e800b7870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00abe0 tx=0x7f0e8c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.888+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 msgr2=0x7f0e9c1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.888+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eaa0 tx=0x7f0e9400eea0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 shutdown_connections 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0e800b53b0 0x7f0e800b7870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e9c10b280 0x7f0e9c1a0c00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e9c10a900 0x7f0e9c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 --2- 172.21.15.67:0/3725474109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e9c104f00 0x7f0e9c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 >> 172.21.15.67:0/3725474109 conn(0x7f0e9c1008e0 msgr2=0x7f0e9c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 shutdown_connections 2024-09-10T22:59:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:25.889+0000 7f0ea3800640 1 -- 172.21.15.67:0/3725474109 wait complete. 2024-09-10T22:59:25.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T22:59:26.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 -- 172.21.15.67:0/3085849886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f410b0a0 msgr2=0x7fd9f41118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:26.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 --2- 172.21.15.67:0/3085849886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41118a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009ca0 tx=0x7fd9ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 -- 172.21.15.67:0/3085849886 shutdown_connections 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 --2- 172.21.15.67:0/3085849886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41118a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 --2- 172.21.15.67:0/3085849886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 --2- 172.21.15.67:0/3085849886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f4104d20 0x7fd9f4105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 -- 172.21.15.67:0/3085849886 >> 172.21.15.67:0/3085849886 conn(0x7fd9f4100740 msgr2=0x7fd9f4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 -- 172.21.15.67:0/3085849886 shutdown_connections 2024-09-10T22:59:26.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.060+0000 7fd9fa400640 1 -- 172.21.15.67:0/3085849886 wait complete. 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.061+0000 7fd9fa400640 1 Processor -- start 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.061+0000 7fd9fa400640 1 -- start start 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4114c20 con 0x7fd9f4104d20 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4114aa0 con 0x7fd9f410a720 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9fa400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4114da0 con 0x7fd9f410b0a0 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:26.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34018/0 (socket says 172.21.15.67:34018) 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 -- 172.21.15.67:0/1719024875 learned_addr learned my addr 172.21.15.67:0/1719024875 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9a00640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f3e00640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 -- 172.21.15.67:0/1719024875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f410b0a0 msgr2=0x7fd9f41a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 -- 172.21.15.67:0/1719024875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 msgr2=0x7fd9f419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.062+0000 7fd9f9000640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f9000640 1 -- 172.21.15.67:0/1719024875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f41a1120 con 0x7fd9f4104d20 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f3e00640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f9000640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c920 tx=0x7fd9e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8007c70 con 0x7fd9f4104d20 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9e8007e10 con 0x7fd9f4104d20 2024-09-10T22:59:26.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8018710 con 0x7fd9f4104d20 2024-09-10T22:59:26.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41a1410 con 0x7fd9f4104d20 2024-09-10T22:59:26.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.063+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41a7cb0 con 0x7fd9f4104d20 2024-09-10T22:59:26.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.064+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4105100 con 0x7fd9f4104d20 2024-09-10T22:59:26.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.067+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd9e800f450 con 0x7fd9f4104d20 2024-09-10T22:59:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.069+0000 7fd9f1600640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 0x7fd9dc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.069+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd9e80e6220 con 0x7fd9f4104d20 2024-09-10T22:59:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.069+0000 7fd9f3e00640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 0x7fd9dc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.070+0000 7fd9f3e00640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 0x7fd9dc0b7870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9f41030e0 tx=0x7fd9e0005d20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:26.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.073+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd9e8014030 con 0x7fd9f4104d20 2024-09-10T22:59:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.291+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd9f4101620 con 0x7fd9dc0b53b0 2024-09-10T22:59:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.302+0000 7fd9f1600640 1 -- 172.21.15.67:0/1719024875 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd9f4101620 con 0x7fd9dc0b53b0 2024-09-10T22:59:26.303 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 msgr2=0x7fd9dc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 0x7fd9dc0b7870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9f41030e0 tx=0x7fd9e0005d20 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 msgr2=0x7fd9f419b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c920 tx=0x7fd9e800cde0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 shutdown_connections 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9dc0b53b0 0x7fd9dc0b7870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9f410b0a0 0x7fd9f41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f410a720 0x7fd9f419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 --2- 172.21.15.67:0/1719024875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9f4104d20 0x7fd9f419b080 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 >> 172.21.15.67:0/1719024875 conn(0x7fd9f4100740 msgr2=0x7fd9f4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 shutdown_connections 2024-09-10T22:59:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:26.305+0000 7fd9fa400640 1 -- 172.21.15.67:0/1719024875 wait complete. 2024-09-10T22:59:26.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-10T22:59:26.385 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-10T22:59:26.396 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T22:59:26.410 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:59:26.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:59:26.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:26 smithi145 ceph-mon[34397]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:26.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:26 smithi145 ceph-mon[34397]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:26.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:26 smithi092 ceph-mon[34223]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:26.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:26 smithi092 ceph-mon[34223]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:26.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:26 smithi067 ceph-mon[30939]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:26.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:26 smithi067 ceph-mon[30939]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 -- 172.21.15.67:0/854899995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 msgr2=0x7eff8c111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 --2- 172.21.15.67:0/854899995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c111b30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7eff84009900 tx=0x7eff8402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 -- 172.21.15.67:0/854899995 shutdown_connections 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 --2- 172.21.15.67:0/854899995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c111b30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 --2- 172.21.15.67:0/854899995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 0x7eff8c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 --2- 172.21.15.67:0/854899995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 0x7eff8c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 -- 172.21.15.67:0/854899995 >> 172.21.15.67:0/854899995 conn(0x7eff8c100a50 msgr2=0x7eff8c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 -- 172.21.15.67:0/854899995 shutdown_connections 2024-09-10T22:59:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.415+0000 7eff92200640 1 -- 172.21.15.67:0/854899995 wait complete. 2024-09-10T22:59:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.416+0000 7eff92200640 1 Processor -- start 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.416+0000 7eff92200640 1 -- start start 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 0x7eff8c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 0x7eff8c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c114e60 con 0x7eff8c10b330 2024-09-10T22:59:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c114ce0 con 0x7eff8c104fa0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff92200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c114fe0 con 0x7eff8c10a9b0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff82a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 0x7eff8c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34040/0 (socket says 172.21.15.67:34040) 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff8be00640 1 -- 172.21.15.67:0/1869291367 learned_addr learned my addr 172.21.15.67:0/1869291367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.417+0000 7eff8b400640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 0x7eff8c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 -- 172.21.15.67:0/1869291367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 msgr2=0x7eff8c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 0x7eff8c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 -- 172.21.15.67:0/1869291367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 msgr2=0x7eff8c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 0x7eff8c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 -- 172.21.15.67:0/1869291367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c1a1360 con 0x7eff8c10b330 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff8be00640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7eff84004770 tx=0x7eff84003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8403f070 con 0x7eff8c10b330 2024-09-10T22:59:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1a15f0 con 0x7eff8c10b330 2024-09-10T22:59:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.418+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff8403bd00 con 0x7eff8c10b330 2024-09-10T22:59:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.419+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84035030 con 0x7eff8c10b330 2024-09-10T22:59:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.419+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a7d10 con 0x7eff8c10b330 2024-09-10T22:59:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.420+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c105380 con 0x7eff8c10b330 2024-09-10T22:59:27.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.422+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7eff84031bd0 con 0x7eff8c10b330 2024-09-10T22:59:27.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.423+0000 7eff88c00640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 0x7eff700b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:27.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.424+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7eff84106c50 con 0x7eff8c10b330 2024-09-10T22:59:27.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.424+0000 7eff8b400640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 0x7eff700b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:27.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.424+0000 7eff8b400640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 0x7eff700b7900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff7c007ec0 tx=0x7eff7c0075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.427+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7eff840cce50 con 0x7eff8c10b330 2024-09-10T22:59:27.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.618+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff8c063590 con 0x7eff700b5440 2024-09-10T22:59:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:27 smithi067 ceph-mon[30939]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T22:59:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:27 smithi067 ceph-mon[30939]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:27 smithi067 ceph-mon[30939]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-10T22:59:27.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.625+0000 7eff88c00640 1 -- 172.21.15.67:0/1869291367 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340792 (secure 0 0 0) 0x7eff8c063590 con 0x7eff700b5440 2024-09-10T22:59:27.630 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:27.632 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:59:27.634 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:27 smithi145 ceph-mon[34397]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T22:59:27.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:27 smithi145 ceph-mon[34397]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:27.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:27 smithi145 ceph-mon[34397]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 msgr2=0x7eff700b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 0x7eff700b7900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff7c007ec0 tx=0x7eff7c0075b0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 msgr2=0x7eff8c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7eff84004770 tx=0x7eff84003cd0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 shutdown_connections 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff700b5440 0x7eff700b7900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff8c10b330 0x7eff8c1a0b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8c10a9b0 0x7eff8c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 --2- 172.21.15.67:0/1869291367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104fa0 0x7eff8c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:27.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.630+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 >> 172.21.15.67:0/1869291367 conn(0x7eff8c100a50 msgr2=0x7eff8c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:27.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.631+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 shutdown_connections 2024-09-10T22:59:27.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:27.631+0000 7eff92200640 1 -- 172.21.15.67:0/1869291367 wait complete. 2024-09-10T22:59:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:27 smithi092 ceph-mon[34223]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T22:59:27.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:27 smithi092 ceph-mon[34223]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:59:27.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:27 smithi092 ceph-mon[34223]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-10T22:59:28.474 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T22:59:26.107278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658544,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":904,"num_read_kb":556,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357152,"kb_used_data":12024,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584672,"statfs":{"total":1151940427776,"available":1151574704128,"internally_reserved":0,"allocated":12312576,"data_stored":8564792,"data_compressed":481138,"data_compressed_allocated":651264,"data_compressed_original":3121152,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1479,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003430"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184942+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:58:34.184942+0000","last_peered":"2024-09-10T22:58:34.184942+0000","last_clean":"2024-09-10T22:58:34.184942+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:58:34.184942+0000","last_undegraded":"2024-09-10T22:58:34.184942+0000","last_fullsized":"2024-09-10T22:58:34.184942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462913+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:58:34.462913+0000","last_peered":"2024-09-10T22:58:34.462913+0000","last_clean":"2024-09-10T22:58:34.462913+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:58:34.462913+0000","last_undegraded":"2024-09-10T22:58:34.462913+0000","last_fullsized":"2024-09-10T22:58:34.462913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188615+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:58:34.188615+0000","last_peered":"2024-09-10T22:58:34.188615+0000","last_clean":"2024-09-10T22:58:34.188615+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:58:34.188615+0000","last_undegraded":"2024-09-10T22:58:34.188615+0000","last_fullsized":"2024-09-10T22:58:34.188615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463180+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:58:34.463180+0000","last_peered":"2024-09-10T22:58:34.463180+0000","last_clean":"2024-09-10T22:58:34.463180+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:58:34.463180+0000","last_undegraded":"2024-09-10T22:58:34.463180+0000","last_fullsized":"2024-09-10T22:58:34.463180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666721+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:58:34.666721+0000","last_peered":"2024-09-10T22:58:34.666721+0000","last_clean":"2024-09-10T22:58:34.666721+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:58:34.666721+0000","last_undegraded":"2024-09-10T22:58:34.666721+0000","last_fullsized":"2024-09-10T22:58:34.666721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187313+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:58:34.187313+0000","last_peered":"2024-09-10T22:58:34.187313+0000","last_clean":"2024-09-10T22:58:34.187313+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:58:34.187313+0000","last_undegraded":"2024-09-10T22:58:34.187313+0000","last_fullsized":"2024-09-10T22:58:34.187313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465001+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:58:34.465001+0000","last_peered":"2024-09-10T22:58:34.465001+0000","last_clean":"2024-09-10T22:58:34.465001+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:58:34.465001+0000","last_undegraded":"2024-09-10T22:58:34.465001+0000","last_fullsized":"2024-09-10T22:58:34.465001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587823+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:58:34.587823+0000","last_peered":"2024-09-10T22:58:34.587823+0000","last_clean":"2024-09-10T22:58:34.587823+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:58:34.587823+0000","last_undegraded":"2024-09-10T22:58:34.587823+0000","last_fullsized":"2024-09-10T22:58:34.587823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188011+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:58:34.188011+0000","last_peered":"2024-09-10T22:58:34.188011+0000","last_clean":"2024-09-10T22:58:34.188011+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:58:34.188011+0000","last_undegraded":"2024-09-10T22:58:34.188011+0000","last_fullsized":"2024-09-10T22:58:34.188011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186395+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:58:34.186395+0000","last_peered":"2024-09-10T22:58:34.186395+0000","last_clean":"2024-09-10T22:58:34.186395+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:58:34.186395+0000","last_undegraded":"2024-09-10T22:58:34.186395+0000","last_fullsized":"2024-09-10T22:58:34.186395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602782+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:58:34.602782+0000","last_peered":"2024-09-10T22:58:34.602782+0000","last_clean":"2024-09-10T22:58:34.602782+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:58:34.602782+0000","last_undegraded":"2024-09-10T22:58:34.602782+0000","last_fullsized":"2024-09-10T22:58:34.602782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187578+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:58:34.187578+0000","last_peered":"2024-09-10T22:58:34.187578+0000","last_clean":"2024-09-10T22:58:34.187578+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:58:34.187578+0000","last_undegraded":"2024-09-10T22:58:34.187578+0000","last_fullsized":"2024-09-10T22:58:34.187578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185664+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:58:34.185664+0000","last_peered":"2024-09-10T22:58:34.185664+0000","last_clean":"2024-09-10T22:58:34.185664+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:58:34.185664+0000","last_undegraded":"2024-09-10T22:58:34.185664+0000","last_fullsized":"2024-09-10T22:58:34.185664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464494+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:58:34.464494+0000","last_peered":"2024-09-10T22:58:34.464494+0000","last_clean":"2024-09-10T22:58:34.464494+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:58:34.464494+0000","last_undegraded":"2024-09-10T22:58:34.464494+0000","last_fullsized":"2024-09-10T22:58:34.464494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187941+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:58:34.187941+0000","last_peered":"2024-09-10T22:58:34.187941+0000","last_clean":"2024-09-10T22:58:34.187941+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:58:34.187941+0000","last_undegraded":"2024-09-10T22:58:34.187941+0000","last_fullsized":"2024-09-10T22:58:34.187941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462839+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:58:34.462839+0000","last_peered":"2024-09-10T22:58:34.462839+0000","last_clean":"2024-09-10T22:58:34.462839+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:58:34.462839+0000","last_undegraded":"2024-09-10T22:58:34.462839+0000","last_fullsized":"2024-09-10T22:58:34.462839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188745+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:58:34.188745+0000","last_peered":"2024-09-10T22:58:34.188745+0000","last_clean":"2024-09-10T22:58:34.188745+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:58:34.188745+0000","last_undegraded":"2024-09-10T22:58:34.188745+0000","last_fullsized":"2024-09-10T22:58:34.188745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187786+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:58:34.187786+0000","last_peered":"2024-09-10T22:58:34.187786+0000","last_clean":"2024-09-10T22:58:34.187786+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:58:34.187786+0000","last_undegraded":"2024-09-10T22:58:34.187786+0000","last_fullsized":"2024-09-10T22:58:34.187786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186083+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:58:34.186083+0000","last_peered":"2024-09-10T22:58:34.186083+0000","last_clean":"2024-09-10T22:58:34.186083+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:58:34.186083+0000","last_undegraded":"2024-09-10T22:58:34.186083+0000","last_fullsized":"2024-09-10T22:58:34.186083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462767+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:58:34.462767+0000","last_peered":"2024-09-10T22:58:34.462767+0000","last_clean":"2024-09-10T22:58:34.462767+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:58:34.462767+0000","last_undegraded":"2024-09-10T22:58:34.462767+0000","last_fullsized":"2024-09-10T22:58:34.462767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184820+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:58:34.184820+0000","last_peered":"2024-09-10T22:58:34.184820+0000","last_clean":"2024-09-10T22:58:34.184820+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:58:34.184820+0000","last_undegraded":"2024-09-10T22:58:34.184820+0000","last_fullsized":"2024-09-10T22:58:34.184820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464201+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:58:34.464201+0000","last_peered":"2024-09-10T22:58:34.464201+0000","last_clean":"2024-09-10T22:58:34.464201+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:58:34.464201+0000","last_undegraded":"2024-09-10T22:58:34.464201+0000","last_fullsized":"2024-09-10T22:58:34.464201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587613+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:58:34.587613+0000","last_peered":"2024-09-10T22:58:34.587613+0000","last_clean":"2024-09-10T22:58:34.587613+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:58:34.587613+0000","last_undegraded":"2024-09-10T22:58:34.587613+0000","last_fullsized":"2024-09-10T22:58:34.587613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466740+0000","last_change":"2024-09-10T22:58:34.466034+0000","last_active":"2024-09-10T22:58:34.466740+0000","last_peered":"2024-09-10T22:58:34.466740+0000","last_clean":"2024-09-10T22:58:34.466740+0000","last_became_active":"2024-09-10T22:58:34.465493+0000","last_became_peered":"2024-09-10T22:58:34.465493+0000","last_unstale":"2024-09-10T22:58:34.466740+0000","last_undegraded":"2024-09-10T22:58:34.466740+0000","last_fullsized":"2024-09-10T22:58:34.466740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:43.360753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666497+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:58:34.666497+0000","last_peered":"2024-09-10T22:58:34.666497+0000","last_clean":"2024-09-10T22:58:34.666497+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:58:34.666497+0000","last_undegraded":"2024-09-10T22:58:34.666497+0000","last_fullsized":"2024-09-10T22:58:34.666497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188622+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:58:34.188622+0000","last_peered":"2024-09-10T22:58:34.188622+0000","last_clean":"2024-09-10T22:58:34.188622+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:58:34.188622+0000","last_undegraded":"2024-09-10T22:58:34.188622+0000","last_fullsized":"2024-09-10T22:58:34.188622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188981+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:58:34.188981+0000","last_peered":"2024-09-10T22:58:34.188981+0000","last_clean":"2024-09-10T22:58:34.188981+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:58:34.188981+0000","last_undegraded":"2024-09-10T22:58:34.188981+0000","last_fullsized":"2024-09-10T22:58:34.188981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463075+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:58:34.463075+0000","last_peered":"2024-09-10T22:58:34.463075+0000","last_clean":"2024-09-10T22:58:34.463075+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:58:34.463075+0000","last_undegraded":"2024-09-10T22:58:34.463075+0000","last_fullsized":"2024-09-10T22:58:34.463075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184759+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:58:34.184759+0000","last_peered":"2024-09-10T22:58:34.184759+0000","last_clean":"2024-09-10T22:58:34.184759+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:58:34.184759+0000","last_undegraded":"2024-09-10T22:58:34.184759+0000","last_fullsized":"2024-09-10T22:58:34.184759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462341+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:58:34.462341+0000","last_peered":"2024-09-10T22:58:34.462341+0000","last_clean":"2024-09-10T22:58:34.462341+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:58:34.462341+0000","last_undegraded":"2024-09-10T22:58:34.462341+0000","last_fullsized":"2024-09-10T22:58:34.462341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186602+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:58:34.186602+0000","last_peered":"2024-09-10T22:58:34.186602+0000","last_clean":"2024-09-10T22:58:34.186602+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:58:34.186602+0000","last_undegraded":"2024-09-10T22:58:34.186602+0000","last_fullsized":"2024-09-10T22:58:34.186602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464970+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:58:34.464970+0000","last_peered":"2024-09-10T22:58:34.464970+0000","last_clean":"2024-09-10T22:58:34.464970+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:58:34.464970+0000","last_undegraded":"2024-09-10T22:58:34.464970+0000","last_fullsized":"2024-09-10T22:58:34.464970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185884+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:58:34.185884+0000","last_peered":"2024-09-10T22:58:34.185884+0000","last_clean":"2024-09-10T22:58:34.185884+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:58:34.185884+0000","last_undegraded":"2024-09-10T22:58:34.185884+0000","last_fullsized":"2024-09-10T22:58:34.185884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189026+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:58:34.189026+0000","last_peered":"2024-09-10T22:58:34.189026+0000","last_clean":"2024-09-10T22:58:34.189026+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:58:34.189026+0000","last_undegraded":"2024-09-10T22:58:34.189026+0000","last_fullsized":"2024-09-10T22:58:34.189026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184648+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:58:34.184648+0000","last_peered":"2024-09-10T22:58:34.184648+0000","last_clean":"2024-09-10T22:58:34.184648+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:58:34.184648+0000","last_undegraded":"2024-09-10T22:58:34.184648+0000","last_fullsized":"2024-09-10T22:58:34.184648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464415+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:58:34.464415+0000","last_peered":"2024-09-10T22:58:34.464415+0000","last_clean":"2024-09-10T22:58:34.464415+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:58:34.464415+0000","last_undegraded":"2024-09-10T22:58:34.464415+0000","last_fullsized":"2024-09-10T22:58:34.464415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188871+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:58:34.188871+0000","last_peered":"2024-09-10T22:58:34.188871+0000","last_clean":"2024-09-10T22:58:34.188871+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:58:34.188871+0000","last_undegraded":"2024-09-10T22:58:34.188871+0000","last_fullsized":"2024-09-10T22:58:34.188871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185784+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:58:34.185784+0000","last_peered":"2024-09-10T22:58:34.185784+0000","last_clean":"2024-09-10T22:58:34.185784+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:58:34.185784+0000","last_undegraded":"2024-09-10T22:58:34.185784+0000","last_fullsized":"2024-09-10T22:58:34.185784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186540+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:58:34.186540+0000","last_peered":"2024-09-10T22:58:34.186540+0000","last_clean":"2024-09-10T22:58:34.186540+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:58:34.186540+0000","last_undegraded":"2024-09-10T22:58:34.186540+0000","last_fullsized":"2024-09-10T22:58:34.186540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602469+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:58:34.602469+0000","last_peered":"2024-09-10T22:58:34.602469+0000","last_clean":"2024-09-10T22:58:34.602469+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:58:34.602469+0000","last_undegraded":"2024-09-10T22:58:34.602469+0000","last_fullsized":"2024-09-10T22:58:34.602469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:23.694786+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:59:23.694786+0000","last_peered":"2024-09-10T22:59:23.694786+0000","last_clean":"2024-09-10T22:59:23.694786+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:59:23.694786+0000","last_undegraded":"2024-09-10T22:59:23.694786+0000","last_fullsized":"2024-09-10T22:59:23.694786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186783+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:58:34.186783+0000","last_peered":"2024-09-10T22:58:34.186783+0000","last_clean":"2024-09-10T22:58:34.186783+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:58:34.186783+0000","last_undegraded":"2024-09-10T22:58:34.186783+0000","last_fullsized":"2024-09-10T22:58:34.186783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.190954+0000","last_change":"2024-09-10T22:58:34.190886+0000","last_active":"2024-09-10T22:58:34.190954+0000","last_peered":"2024-09-10T22:58:34.190954+0000","last_clean":"2024-09-10T22:58:34.190954+0000","last_became_active":"2024-09-10T22:58:34.190534+0000","last_became_peered":"2024-09-10T22:58:34.190534+0000","last_unstale":"2024-09-10T22:58:34.190954+0000","last_undegraded":"2024-09-10T22:58:34.190954+0000","last_fullsized":"2024-09-10T22:58:34.190954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:07.888931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185571+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:58:34.185571+0000","last_peered":"2024-09-10T22:58:34.185571+0000","last_clean":"2024-09-10T22:58:34.185571+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:58:34.185571+0000","last_undegraded":"2024-09-10T22:58:34.185571+0000","last_fullsized":"2024-09-10T22:58:34.185571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.603153+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:58:34.603153+0000","last_peered":"2024-09-10T22:58:34.603153+0000","last_clean":"2024-09-10T22:58:34.603153+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:58:34.603153+0000","last_undegraded":"2024-09-10T22:58:34.603153+0000","last_fullsized":"2024-09-10T22:58:34.603153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184347+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:58:34.184347+0000","last_peered":"2024-09-10T22:58:34.184347+0000","last_clean":"2024-09-10T22:58:34.184347+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:58:34.184347+0000","last_undegraded":"2024-09-10T22:58:34.184347+0000","last_fullsized":"2024-09-10T22:58:34.184347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188099+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:58:34.188099+0000","last_peered":"2024-09-10T22:58:34.188099+0000","last_clean":"2024-09-10T22:58:34.188099+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:58:34.188099+0000","last_undegraded":"2024-09-10T22:58:34.188099+0000","last_fullsized":"2024-09-10T22:58:34.188099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185870+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:58:34.185870+0000","last_peered":"2024-09-10T22:58:34.185870+0000","last_clean":"2024-09-10T22:58:34.185870+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:58:34.185870+0000","last_undegraded":"2024-09-10T22:58:34.185870+0000","last_fullsized":"2024-09-10T22:58:34.185870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186444+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:58:34.186444+0000","last_peered":"2024-09-10T22:58:34.186444+0000","last_clean":"2024-09-10T22:58:34.186444+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:58:34.186444+0000","last_undegraded":"2024-09-10T22:58:34.186444+0000","last_fullsized":"2024-09-10T22:58:34.186444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186412+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:58:34.186412+0000","last_peered":"2024-09-10T22:58:34.186412+0000","last_clean":"2024-09-10T22:58:34.186412+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:58:34.186412+0000","last_undegraded":"2024-09-10T22:58:34.186412+0000","last_fullsized":"2024-09-10T22:58:34.186412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189203+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:58:34.189203+0000","last_peered":"2024-09-10T22:58:34.189203+0000","last_clean":"2024-09-10T22:58:34.189203+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:58:34.189203+0000","last_undegraded":"2024-09-10T22:58:34.189203+0000","last_fullsized":"2024-09-10T22:58:34.189203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587205+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:58:34.587205+0000","last_peered":"2024-09-10T22:58:34.587205+0000","last_clean":"2024-09-10T22:58:34.587205+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:58:34.587205+0000","last_undegraded":"2024-09-10T22:58:34.587205+0000","last_fullsized":"2024-09-10T22:58:34.587205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185988+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:58:34.185988+0000","last_peered":"2024-09-10T22:58:34.185988+0000","last_clean":"2024-09-10T22:58:34.185988+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:58:34.185988+0000","last_undegraded":"2024-09-10T22:58:34.185988+0000","last_fullsized":"2024-09-10T22:58:34.185988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602862+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:58:34.602862+0000","last_peered":"2024-09-10T22:58:34.602862+0000","last_clean":"2024-09-10T22:58:34.602862+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:58:34.602862+0000","last_undegraded":"2024-09-10T22:58:34.602862+0000","last_fullsized":"2024-09-10T22:58:34.602862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666303+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:58:34.666303+0000","last_peered":"2024-09-10T22:58:34.666303+0000","last_clean":"2024-09-10T22:58:34.666303+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:58:34.666303+0000","last_undegraded":"2024-09-10T22:58:34.666303+0000","last_fullsized":"2024-09-10T22:58:34.666303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186147+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:58:34.186147+0000","last_peered":"2024-09-10T22:58:34.186147+0000","last_clean":"2024-09-10T22:58:34.186147+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:58:34.186147+0000","last_undegraded":"2024-09-10T22:58:34.186147+0000","last_fullsized":"2024-09-10T22:58:34.186147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187257+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:58:34.187257+0000","last_peered":"2024-09-10T22:58:34.187257+0000","last_clean":"2024-09-10T22:58:34.187257+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:58:34.187257+0000","last_undegraded":"2024-09-10T22:58:34.187257+0000","last_fullsized":"2024-09-10T22:58:34.187257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.601976+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:58:34.601976+0000","last_peered":"2024-09-10T22:58:34.601976+0000","last_clean":"2024-09-10T22:58:34.601976+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:58:34.601976+0000","last_undegraded":"2024-09-10T22:58:34.601976+0000","last_fullsized":"2024-09-10T22:58:34.601976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186276+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:58:34.186276+0000","last_peered":"2024-09-10T22:58:34.186276+0000","last_clean":"2024-09-10T22:58:34.186276+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:58:34.186276+0000","last_undegraded":"2024-09-10T22:58:34.186276+0000","last_fullsized":"2024-09-10T22:58:34.186276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465088+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:58:34.465088+0000","last_peered":"2024-09-10T22:58:34.465088+0000","last_clean":"2024-09-10T22:58:34.465088+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:58:34.465088+0000","last_undegraded":"2024-09-10T22:58:34.465088+0000","last_fullsized":"2024-09-10T22:58:34.465088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602472+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:58:34.602472+0000","last_peered":"2024-09-10T22:58:34.602472+0000","last_clean":"2024-09-10T22:58:34.602472+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:58:34.602472+0000","last_undegraded":"2024-09-10T22:58:34.602472+0000","last_fullsized":"2024-09-10T22:58:34.602472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602600+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:58:34.602600+0000","last_peered":"2024-09-10T22:58:34.602600+0000","last_clean":"2024-09-10T22:58:34.602600+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:58:34.602600+0000","last_undegraded":"2024-09-10T22:58:34.602600+0000","last_fullsized":"2024-09-10T22:58:34.602600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185766+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:58:34.185766+0000","last_peered":"2024-09-10T22:58:34.185766+0000","last_clean":"2024-09-10T22:58:34.185766+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:58:34.185766+0000","last_undegraded":"2024-09-10T22:58:34.185766+0000","last_fullsized":"2024-09-10T22:58:34.185766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466980+0000","last_change":"2024-09-10T22:58:34.466291+0000","last_active":"2024-09-10T22:58:34.466980+0000","last_peered":"2024-09-10T22:58:34.466980+0000","last_clean":"2024-09-10T22:58:34.466980+0000","last_became_active":"2024-09-10T22:58:34.465710+0000","last_became_peered":"2024-09-10T22:58:34.465710+0000","last_unstale":"2024-09-10T22:58:34.466980+0000","last_undegraded":"2024-09-10T22:58:34.466980+0000","last_fullsized":"2024-09-10T22:58:34.466980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:32.445364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185427+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:58:34.185427+0000","last_peered":"2024-09-10T22:58:34.185427+0000","last_clean":"2024-09-10T22:58:34.185427+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:58:34.185427+0000","last_undegraded":"2024-09-10T22:58:34.185427+0000","last_fullsized":"2024-09-10T22:58:34.185427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:23.695454+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:59:23.695454+0000","last_peered":"2024-09-10T22:59:23.695454+0000","last_clean":"2024-09-10T22:59:23.695454+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:59:23.695454+0000","last_undegraded":"2024-09-10T22:59:23.695454+0000","last_fullsized":"2024-09-10T22:59:23.695454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184894+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:58:34.184894+0000","last_peered":"2024-09-10T22:58:34.184894+0000","last_clean":"2024-09-10T22:58:34.184894+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:58:34.184894+0000","last_undegraded":"2024-09-10T22:58:34.184894+0000","last_fullsized":"2024-09-10T22:58:34.184894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185942+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:58:34.185942+0000","last_peered":"2024-09-10T22:58:34.185942+0000","last_clean":"2024-09-10T22:58:34.185942+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:58:34.185942+0000","last_undegraded":"2024-09-10T22:58:34.185942+0000","last_fullsized":"2024-09-10T22:58:34.185942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602689+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:58:34.602689+0000","last_peered":"2024-09-10T22:58:34.602689+0000","last_clean":"2024-09-10T22:58:34.602689+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:58:34.602689+0000","last_undegraded":"2024-09-10T22:58:34.602689+0000","last_fullsized":"2024-09-10T22:58:34.602689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185960+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:58:34.185960+0000","last_peered":"2024-09-10T22:58:34.185960+0000","last_clean":"2024-09-10T22:58:34.185960+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:58:34.185960+0000","last_undegraded":"2024-09-10T22:58:34.185960+0000","last_fullsized":"2024-09-10T22:58:34.185960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186578+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:58:34.186578+0000","last_peered":"2024-09-10T22:58:34.186578+0000","last_clean":"2024-09-10T22:58:34.186578+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:58:34.186578+0000","last_undegraded":"2024-09-10T22:58:34.186578+0000","last_fullsized":"2024-09-10T22:58:34.186578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188921+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:58:34.188921+0000","last_peered":"2024-09-10T22:58:34.188921+0000","last_clean":"2024-09-10T22:58:34.188921+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:58:34.188921+0000","last_undegraded":"2024-09-10T22:58:34.188921+0000","last_fullsized":"2024-09-10T22:58:34.188921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602556+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:58:34.602556+0000","last_peered":"2024-09-10T22:58:34.602556+0000","last_clean":"2024-09-10T22:58:34.602556+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:58:34.602556+0000","last_undegraded":"2024-09-10T22:58:34.602556+0000","last_fullsized":"2024-09-10T22:58:34.602556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464901+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:58:34.464901+0000","last_peered":"2024-09-10T22:58:34.464901+0000","last_clean":"2024-09-10T22:58:34.464901+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:58:34.464901+0000","last_undegraded":"2024-09-10T22:58:34.464901+0000","last_fullsized":"2024-09-10T22:58:34.464901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:25.700231+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:59:25.700231+0000","last_peered":"2024-09-10T22:59:25.700231+0000","last_clean":"2024-09-10T22:59:25.700231+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:59:25.700231+0000","last_undegraded":"2024-09-10T22:59:25.700231+0000","last_fullsized":"2024-09-10T22:59:25.700231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62814,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":277,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.604692+0000","last_change":"2024-09-10T22:58:34.604157+0000","last_active":"2024-09-10T22:58:34.604692+0000","last_peered":"2024-09-10T22:58:34.604692+0000","last_clean":"2024-09-10T22:58:34.604692+0000","last_became_active":"2024-09-10T22:58:34.603682+0000","last_became_peered":"2024-09-10T22:58:34.603682+0000","last_unstale":"2024-09-10T22:58:34.604692+0000","last_undegraded":"2024-09-10T22:58:34.604692+0000","last_fullsized":"2024-09-10T22:58:34.604692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:44.610556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587420+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:58:34.587420+0000","last_peered":"2024-09-10T22:58:34.587420+0000","last_clean":"2024-09-10T22:58:34.587420+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:58:34.587420+0000","last_undegraded":"2024-09-10T22:58:34.587420+0000","last_fullsized":"2024-09-10T22:58:34.587420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602989+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:58:34.602989+0000","last_peered":"2024-09-10T22:58:34.602989+0000","last_clean":"2024-09-10T22:58:34.602989+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:58:34.602989+0000","last_undegraded":"2024-09-10T22:58:34.602989+0000","last_fullsized":"2024-09-10T22:58:34.602989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.589489+0000","last_change":"2024-09-10T22:58:34.588867+0000","last_active":"2024-09-10T22:58:34.589489+0000","last_peered":"2024-09-10T22:58:34.589489+0000","last_clean":"2024-09-10T22:58:34.589489+0000","last_became_active":"2024-09-10T22:58:34.588410+0000","last_became_peered":"2024-09-10T22:58:34.588410+0000","last_unstale":"2024-09-10T22:58:34.589489+0000","last_undegraded":"2024-09-10T22:58:34.589489+0000","last_fullsized":"2024-09-10T22:58:34.589489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:24.765755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666891+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:58:34.666891+0000","last_peered":"2024-09-10T22:58:34.666891+0000","last_clean":"2024-09-10T22:58:34.666891+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:58:34.666891+0000","last_undegraded":"2024-09-10T22:58:34.666891+0000","last_fullsized":"2024-09-10T22:58:34.666891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185860+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:58:34.185860+0000","last_peered":"2024-09-10T22:58:34.185860+0000","last_clean":"2024-09-10T22:58:34.185860+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:58:34.185860+0000","last_undegraded":"2024-09-10T22:58:34.185860+0000","last_fullsized":"2024-09-10T22:58:34.185860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184137+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:58:34.184137+0000","last_peered":"2024-09-10T22:58:34.184137+0000","last_clean":"2024-09-10T22:58:34.184137+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:58:34.184137+0000","last_undegraded":"2024-09-10T22:58:34.184137+0000","last_fullsized":"2024-09-10T22:58:34.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:24.698215+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:59:24.698215+0000","last_peered":"2024-09-10T22:59:24.698215+0000","last_clean":"2024-09-10T22:59:24.698215+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:59:24.698215+0000","last_undegraded":"2024-09-10T22:59:24.698215+0000","last_fullsized":"2024-09-10T22:59:24.698215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462520+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:58:34.462520+0000","last_peered":"2024-09-10T22:58:34.462520+0000","last_clean":"2024-09-10T22:58:34.462520+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:58:34.462520+0000","last_undegraded":"2024-09-10T22:58:34.462520+0000","last_fullsized":"2024-09-10T22:58:34.462520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666395+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:58:34.666395+0000","last_peered":"2024-09-10T22:58:34.666395+0000","last_clean":"2024-09-10T22:58:34.666395+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:58:34.666395+0000","last_undegraded":"2024-09-10T22:58:34.666395+0000","last_fullsized":"2024-09-10T22:58:34.666395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184232+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:58:34.184232+0000","last_peered":"2024-09-10T22:58:34.184232+0000","last_clean":"2024-09-10T22:58:34.184232+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:58:34.184232+0000","last_undegraded":"2024-09-10T22:58:34.184232+0000","last_fullsized":"2024-09-10T22:58:34.184232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602683+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:58:34.602683+0000","last_peered":"2024-09-10T22:58:34.602683+0000","last_clean":"2024-09-10T22:58:34.602683+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:58:34.602683+0000","last_undegraded":"2024-09-10T22:58:34.602683+0000","last_fullsized":"2024-09-10T22:58:34.602683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186357+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:58:34.186357+0000","last_peered":"2024-09-10T22:58:34.186357+0000","last_clean":"2024-09-10T22:58:34.186357+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:58:34.186357+0000","last_undegraded":"2024-09-10T22:58:34.186357+0000","last_fullsized":"2024-09-10T22:58:34.186357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462533+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:58:34.462533+0000","last_peered":"2024-09-10T22:58:34.462533+0000","last_clean":"2024-09-10T22:58:34.462533+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:58:34.462533+0000","last_undegraded":"2024-09-10T22:58:34.462533+0000","last_fullsized":"2024-09-10T22:58:34.462533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186324+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:58:34.186324+0000","last_peered":"2024-09-10T22:58:34.186324+0000","last_clean":"2024-09-10T22:58:34.186324+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:58:34.186324+0000","last_undegraded":"2024-09-10T22:58:34.186324+0000","last_fullsized":"2024-09-10T22:58:34.186324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189561+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:58:34.189561+0000","last_peered":"2024-09-10T22:58:34.189561+0000","last_clean":"2024-09-10T22:58:34.189561+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:58:34.189561+0000","last_undegraded":"2024-09-10T22:58:34.189561+0000","last_fullsized":"2024-09-10T22:58:34.189561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602398+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:58:34.602398+0000","last_peered":"2024-09-10T22:58:34.602398+0000","last_clean":"2024-09-10T22:58:34.602398+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:58:34.602398+0000","last_undegraded":"2024-09-10T22:58:34.602398+0000","last_fullsized":"2024-09-10T22:58:34.602398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188728+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:58:34.188728+0000","last_peered":"2024-09-10T22:58:34.188728+0000","last_clean":"2024-09-10T22:58:34.188728+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:58:34.188728+0000","last_undegraded":"2024-09-10T22:58:34.188728+0000","last_fullsized":"2024-09-10T22:58:34.188728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:22.694095+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:59:22.694095+0000","last_peered":"2024-09-10T22:59:22.694095+0000","last_clean":"2024-09-10T22:59:22.694095+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:59:22.694095+0000","last_undegraded":"2024-09-10T22:59:22.694095+0000","last_fullsized":"2024-09-10T22:59:22.694095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464546+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:58:34.464546+0000","last_peered":"2024-09-10T22:58:34.464546+0000","last_clean":"2024-09-10T22:58:34.464546+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:58:34.464546+0000","last_undegraded":"2024-09-10T22:58:34.464546+0000","last_fullsized":"2024-09-10T22:58:34.464546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189292+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:58:34.189292+0000","last_peered":"2024-09-10T22:58:34.189292+0000","last_clean":"2024-09-10T22:58:34.189292+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:58:34.189292+0000","last_undegraded":"2024-09-10T22:58:34.189292+0000","last_fullsized":"2024-09-10T22:58:34.189292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185970+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:58:34.185970+0000","last_peered":"2024-09-10T22:58:34.185970+0000","last_clean":"2024-09-10T22:58:34.185970+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:58:34.185970+0000","last_undegraded":"2024-09-10T22:58:34.185970+0000","last_fullsized":"2024-09-10T22:58:34.185970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189242+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:58:34.189242+0000","last_peered":"2024-09-10T22:58:34.189242+0000","last_clean":"2024-09-10T22:58:34.189242+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:58:34.189242+0000","last_undegraded":"2024-09-10T22:58:34.189242+0000","last_fullsized":"2024-09-10T22:58:34.189242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188923+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:58:34.188923+0000","last_peered":"2024-09-10T22:58:34.188923+0000","last_clean":"2024-09-10T22:58:34.188923+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:58:34.188923+0000","last_undegraded":"2024-09-10T22:58:34.188923+0000","last_fullsized":"2024-09-10T22:58:34.188923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666291+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:58:34.666291+0000","last_peered":"2024-09-10T22:58:34.666291+0000","last_clean":"2024-09-10T22:58:34.666291+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:58:34.666291+0000","last_undegraded":"2024-09-10T22:58:34.666291+0000","last_fullsized":"2024-09-10T22:58:34.666291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186557+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:58:34.186557+0000","last_peered":"2024-09-10T22:58:34.186557+0000","last_clean":"2024-09-10T22:58:34.186557+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:58:34.186557+0000","last_undegraded":"2024-09-10T22:58:34.186557+0000","last_fullsized":"2024-09-10T22:58:34.186557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186283+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:58:34.186283+0000","last_peered":"2024-09-10T22:58:34.186283+0000","last_clean":"2024-09-10T22:58:34.186283+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:58:34.186283+0000","last_undegraded":"2024-09-10T22:58:34.186283+0000","last_fullsized":"2024-09-10T22:58:34.186283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189258+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:58:34.189258+0000","last_peered":"2024-09-10T22:58:34.189258+0000","last_clean":"2024-09-10T22:58:34.189258+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:58:34.189258+0000","last_undegraded":"2024-09-10T22:58:34.189258+0000","last_fullsized":"2024-09-10T22:58:34.189258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462967+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:58:34.462967+0000","last_peered":"2024-09-10T22:58:34.462967+0000","last_clean":"2024-09-10T22:58:34.462967+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:58:34.462967+0000","last_undegraded":"2024-09-10T22:58:34.462967+0000","last_fullsized":"2024-09-10T22:58:34.462967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602043+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:58:34.602043+0000","last_peered":"2024-09-10T22:58:34.602043+0000","last_clean":"2024-09-10T22:58:34.602043+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:58:34.602043+0000","last_undegraded":"2024-09-10T22:58:34.602043+0000","last_fullsized":"2024-09-10T22:58:34.602043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462583+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:58:34.462583+0000","last_peered":"2024-09-10T22:58:34.462583+0000","last_clean":"2024-09-10T22:58:34.462583+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:58:34.462583+0000","last_undegraded":"2024-09-10T22:58:34.462583+0000","last_fullsized":"2024-09-10T22:58:34.462583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186971+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:58:34.186971+0000","last_peered":"2024-09-10T22:58:34.186971+0000","last_clean":"2024-09-10T22:58:34.186971+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:58:34.186971+0000","last_undegraded":"2024-09-10T22:58:34.186971+0000","last_fullsized":"2024-09-10T22:58:34.186971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186021+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:58:34.186021+0000","last_peered":"2024-09-10T22:58:34.186021+0000","last_clean":"2024-09-10T22:58:34.186021+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:58:34.186021+0000","last_undegraded":"2024-09-10T22:58:34.186021+0000","last_fullsized":"2024-09-10T22:58:34.186021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189081+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:58:34.189081+0000","last_peered":"2024-09-10T22:58:34.189081+0000","last_clean":"2024-09-10T22:58:34.189081+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:58:34.189081+0000","last_undegraded":"2024-09-10T22:58:34.189081+0000","last_fullsized":"2024-09-10T22:58:34.189081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184788+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:58:34.184788+0000","last_peered":"2024-09-10T22:58:34.184788+0000","last_clean":"2024-09-10T22:58:34.184788+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:58:34.184788+0000","last_undegraded":"2024-09-10T22:58:34.184788+0000","last_fullsized":"2024-09-10T22:58:34.184788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186656+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:58:34.186656+0000","last_peered":"2024-09-10T22:58:34.186656+0000","last_clean":"2024-09-10T22:58:34.186656+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:58:34.186656+0000","last_undegraded":"2024-09-10T22:58:34.186656+0000","last_fullsized":"2024-09-10T22:58:34.186656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587130+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:58:34.587130+0000","last_peered":"2024-09-10T22:58:34.587130+0000","last_clean":"2024-09-10T22:58:34.587130+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:58:34.587130+0000","last_undegraded":"2024-09-10T22:58:34.587130+0000","last_fullsized":"2024-09-10T22:58:34.587130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666125+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:58:34.666125+0000","last_peered":"2024-09-10T22:58:34.666125+0000","last_clean":"2024-09-10T22:58:34.666125+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:58:34.666125+0000","last_undegraded":"2024-09-10T22:58:34.666125+0000","last_fullsized":"2024-09-10T22:58:34.666125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666081+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:58:34.666081+0000","last_peered":"2024-09-10T22:58:34.666081+0000","last_clean":"2024-09-10T22:58:34.666081+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:58:34.666081+0000","last_undegraded":"2024-09-10T22:58:34.666081+0000","last_fullsized":"2024-09-10T22:58:34.666081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186817+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:58:34.186817+0000","last_peered":"2024-09-10T22:58:34.186817+0000","last_clean":"2024-09-10T22:58:34.186817+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:58:34.186817+0000","last_undegraded":"2024-09-10T22:58:34.186817+0000","last_fullsized":"2024-09-10T22:58:34.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188049+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:58:34.188049+0000","last_peered":"2024-09-10T22:58:34.188049+0000","last_clean":"2024-09-10T22:58:34.188049+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:58:34.188049+0000","last_undegraded":"2024-09-10T22:58:34.188049+0000","last_fullsized":"2024-09-10T22:58:34.188049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186351+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:58:34.186351+0000","last_peered":"2024-09-10T22:58:34.186351+0000","last_clean":"2024-09-10T22:58:34.186351+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:58:34.186351+0000","last_undegraded":"2024-09-10T22:58:34.186351+0000","last_fullsized":"2024-09-10T22:58:34.186351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189733+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:58:34.189733+0000","last_peered":"2024-09-10T22:58:34.189733+0000","last_clean":"2024-09-10T22:58:34.189733+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:58:34.189733+0000","last_undegraded":"2024-09-10T22:58:34.189733+0000","last_fullsized":"2024-09-10T22:58:34.189733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587752+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:58:34.587752+0000","last_peered":"2024-09-10T22:58:34.587752+0000","last_clean":"2024-09-10T22:58:34.587752+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:58:34.587752+0000","last_undegraded":"2024-09-10T22:58:34.587752+0000","last_fullsized":"2024-09-10T22:58:34.587752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462478+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:58:34.462478+0000","last_peered":"2024-09-10T22:58:34.462478+0000","last_clean":"2024-09-10T22:58:34.462478+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:58:34.462478+0000","last_undegraded":"2024-09-10T22:58:34.462478+0000","last_fullsized":"2024-09-10T22:58:34.462478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185517+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:58:34.185517+0000","last_peered":"2024-09-10T22:58:34.185517+0000","last_clean":"2024-09-10T22:58:34.185517+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:58:34.185517+0000","last_undegraded":"2024-09-10T22:58:34.185517+0000","last_fullsized":"2024-09-10T22:58:34.185517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189448+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:58:34.189448+0000","last_peered":"2024-09-10T22:58:34.189448+0000","last_clean":"2024-09-10T22:58:34.189448+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:58:34.189448+0000","last_undegraded":"2024-09-10T22:58:34.189448+0000","last_fullsized":"2024-09-10T22:58:34.189448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464386+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:58:34.464386+0000","last_peered":"2024-09-10T22:58:34.464386+0000","last_clean":"2024-09-10T22:58:34.464386+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:58:34.464386+0000","last_undegraded":"2024-09-10T22:58:34.464386+0000","last_fullsized":"2024-09-10T22:58:34.464386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187920+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:58:34.187920+0000","last_peered":"2024-09-10T22:58:34.187920+0000","last_clean":"2024-09-10T22:58:34.187920+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:58:34.187920+0000","last_undegraded":"2024-09-10T22:58:34.187920+0000","last_fullsized":"2024-09-10T22:58:34.187920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666406+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:58:34.666406+0000","last_peered":"2024-09-10T22:58:34.666406+0000","last_clean":"2024-09-10T22:58:34.666406+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:58:34.666406+0000","last_undegraded":"2024-09-10T22:58:34.666406+0000","last_fullsized":"2024-09-10T22:58:34.666406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666421+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:58:34.666421+0000","last_peered":"2024-09-10T22:58:34.666421+0000","last_clean":"2024-09-10T22:58:34.666421+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:58:34.666421+0000","last_undegraded":"2024-09-10T22:58:34.666421+0000","last_fullsized":"2024-09-10T22:58:34.666421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186517+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:58:34.186517+0000","last_peered":"2024-09-10T22:58:34.186517+0000","last_clean":"2024-09-10T22:58:34.186517+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:58:34.186517+0000","last_undegraded":"2024-09-10T22:58:34.186517+0000","last_fullsized":"2024-09-10T22:58:34.186517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67942,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":823,"num_read_kb":487,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":507904,"data_stored":990818,"data_compressed":37363,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612661,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518482,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808891,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1179648,"data_stored":1562721,"data_compressed":83571,"data_compressed_allocated":110592,"data_compressed_original":1040384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":522702,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518414,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397580,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1115562,"data_compressed":65480,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561107,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":513782,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1110657,"data_compressed":65480,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":519037,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117092,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":513976,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":577122,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":577167,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":999424,"data_stored":515170,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19272,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":458661,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:59:28.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:59:28.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:28 smithi145 ceph-mon[34397]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:28.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:28 smithi092 ceph-mon[34223]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:28.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:28 smithi067 ceph-mon[30939]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:28.861 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:29.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 -- 172.21.15.67:0/3612763852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48102c10 msgr2=0x7f4e48102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 --2- 172.21.15.67:0/3612763852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48102c10 0x7f4e48102ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009900 tx=0x7f4e3802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 -- 172.21.15.67:0/3612763852 shutdown_connections 2024-09-10T22:59:29.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 --2- 172.21.15.67:0/3612763852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 0x7f4e4810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 --2- 172.21.15.67:0/3612763852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48108620 0x7f4e48108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 --2- 172.21.15.67:0/3612763852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48102c10 0x7f4e48102ff0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 -- 172.21.15.67:0/3612763852 >> 172.21.15.67:0/3612763852 conn(0x7f4e480fe660 msgr2=0x7f4e48100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 -- 172.21.15.67:0/3612763852 shutdown_connections 2024-09-10T22:59:29.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.530+0000 7f4e47200640 1 -- 172.21.15.67:0/3612763852 wait complete. 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 Processor -- start 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 -- start start 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 0x7f4e48198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 0x7f4e48199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 0x7f4e4819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48112ae0 con 0x7f4e48108620 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48112960 con 0x7f4e48102c10 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.531+0000 7f4e47200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48112c60 con 0x7f4e48108fa0 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 0x7f4e48198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 0x7f4e48198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52142/0 (socket says 172.21.15.67:52142) 2024-09-10T22:59:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45e00640 1 -- 172.21.15.67:0/4066944681 learned_addr learned my addr 172.21.15.67:0/4066944681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 0x7f4e48199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e46800640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 0x7f4e4819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 -- 172.21.15.67:0/4066944681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 msgr2=0x7f4e4819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 0x7f4e4819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 -- 172.21.15.67:0/4066944681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 msgr2=0x7f4e48198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 0x7f4e48198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.532+0000 7f4e45400640 1 -- 172.21.15.67:0/4066944681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4819f080 con 0x7f4e48108620 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e45400640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 0x7f4e48199510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4e34005fd0 tx=0x7f4e3400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34009c30 con 0x7f4e48108620 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e340044c0 con 0x7f4e48108620 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4819f370 con 0x7f4e48108620 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003d70 con 0x7f4e48108620 2024-09-10T22:59:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.533+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481a5c00 con 0x7f4e48108620 2024-09-10T22:59:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.534+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e48103ec0 con 0x7f4e48108620 2024-09-10T22:59:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.536+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f4e34004000 con 0x7f4e48108620 2024-09-10T22:59:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.538+0000 7f4e3ea00640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 0x7f4e2c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.538+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f4e340e5ff0 con 0x7f4e48108620 2024-09-10T22:59:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.539+0000 7f4e45e00640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 0x7f4e2c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.539+0000 7f4e45e00640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 0x7f4e2c0b79a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.542+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e34017020 con 0x7f4e48108620 2024-09-10T22:59:29.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:29 smithi145 ceph-mon[34397]: pgmap v405: 129 pgs: 129 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-09-10T22:59:29.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.735+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e48063590 con 0x7f4e2c0b54e0 2024-09-10T22:59:29.736 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:29 smithi067 ceph-mon[30939]: pgmap v405: 129 pgs: 129 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-09-10T22:59:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.742+0000 7f4e3ea00640 1 -- 172.21.15.67:0/4066944681 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340841 (secure 0 0 0) 0x7f4e48063590 con 0x7f4e2c0b54e0 2024-09-10T22:59:29.747 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 msgr2=0x7f4e2c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 0x7f4e2c0b79a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38009560 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 msgr2=0x7f4e48199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 0x7f4e48199510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4e34005fd0 tx=0x7f4e3400dde0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 shutdown_connections 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4e2c0b54e0 0x7f4e2c0b79a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e48108fa0 0x7f4e4819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e48108620 0x7f4e48199510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 --2- 172.21.15.67:0/4066944681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e48102c10 0x7f4e48198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 >> 172.21.15.67:0/4066944681 conn(0x7f4e480fe660 msgr2=0x7f4e48100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 shutdown_connections 2024-09-10T22:59:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:29.747+0000 7f4e47200640 1 -- 172.21.15.67:0/4066944681 wait complete. 2024-09-10T22:59:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:29 smithi092 ceph-mon[34223]: pgmap v405: 129 pgs: 129 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-09-10T22:59:30.611 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T22:59:28.107839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1051760,"num_objects":65,"num_object_clones":0,"num_object_copies":197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":1206,"num_read_kb":824,"num_write":464,"num_write_kb":3498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":835,"ondisk_log_size":835,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357168,"kb_used_data":12048,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584656,"statfs":{"total":1151940427776,"available":1151574687744,"internally_reserved":0,"allocated":12337152,"data_stored":8958008,"data_compressed":500427,"data_compressed_allocated":675840,"data_compressed_original":3514368,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":394695,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":364,"num_read_kb":300,"num_write":164,"num_write_kb":1224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003363"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184942+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:58:34.184942+0000","last_peered":"2024-09-10T22:58:34.184942+0000","last_clean":"2024-09-10T22:58:34.184942+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:58:34.184942+0000","last_undegraded":"2024-09-10T22:58:34.184942+0000","last_fullsized":"2024-09-10T22:58:34.184942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462913+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:58:34.462913+0000","last_peered":"2024-09-10T22:58:34.462913+0000","last_clean":"2024-09-10T22:58:34.462913+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:58:34.462913+0000","last_undegraded":"2024-09-10T22:58:34.462913+0000","last_fullsized":"2024-09-10T22:58:34.462913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188615+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:58:34.188615+0000","last_peered":"2024-09-10T22:58:34.188615+0000","last_clean":"2024-09-10T22:58:34.188615+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:58:34.188615+0000","last_undegraded":"2024-09-10T22:58:34.188615+0000","last_fullsized":"2024-09-10T22:58:34.188615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463180+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:58:34.463180+0000","last_peered":"2024-09-10T22:58:34.463180+0000","last_clean":"2024-09-10T22:58:34.463180+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:58:34.463180+0000","last_undegraded":"2024-09-10T22:58:34.463180+0000","last_fullsized":"2024-09-10T22:58:34.463180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666721+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:58:34.666721+0000","last_peered":"2024-09-10T22:58:34.666721+0000","last_clean":"2024-09-10T22:58:34.666721+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:58:34.666721+0000","last_undegraded":"2024-09-10T22:58:34.666721+0000","last_fullsized":"2024-09-10T22:58:34.666721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187313+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:58:34.187313+0000","last_peered":"2024-09-10T22:58:34.187313+0000","last_clean":"2024-09-10T22:58:34.187313+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:58:34.187313+0000","last_undegraded":"2024-09-10T22:58:34.187313+0000","last_fullsized":"2024-09-10T22:58:34.187313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465001+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:58:34.465001+0000","last_peered":"2024-09-10T22:58:34.465001+0000","last_clean":"2024-09-10T22:58:34.465001+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:58:34.465001+0000","last_undegraded":"2024-09-10T22:58:34.465001+0000","last_fullsized":"2024-09-10T22:58:34.465001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587823+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:58:34.587823+0000","last_peered":"2024-09-10T22:58:34.587823+0000","last_clean":"2024-09-10T22:58:34.587823+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:58:34.587823+0000","last_undegraded":"2024-09-10T22:58:34.587823+0000","last_fullsized":"2024-09-10T22:58:34.587823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188011+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:58:34.188011+0000","last_peered":"2024-09-10T22:58:34.188011+0000","last_clean":"2024-09-10T22:58:34.188011+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:58:34.188011+0000","last_undegraded":"2024-09-10T22:58:34.188011+0000","last_fullsized":"2024-09-10T22:58:34.188011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186395+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:58:34.186395+0000","last_peered":"2024-09-10T22:58:34.186395+0000","last_clean":"2024-09-10T22:58:34.186395+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:58:34.186395+0000","last_undegraded":"2024-09-10T22:58:34.186395+0000","last_fullsized":"2024-09-10T22:58:34.186395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602782+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:58:34.602782+0000","last_peered":"2024-09-10T22:58:34.602782+0000","last_clean":"2024-09-10T22:58:34.602782+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:58:34.602782+0000","last_undegraded":"2024-09-10T22:58:34.602782+0000","last_fullsized":"2024-09-10T22:58:34.602782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187578+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:58:34.187578+0000","last_peered":"2024-09-10T22:58:34.187578+0000","last_clean":"2024-09-10T22:58:34.187578+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:58:34.187578+0000","last_undegraded":"2024-09-10T22:58:34.187578+0000","last_fullsized":"2024-09-10T22:58:34.187578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185664+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:58:34.185664+0000","last_peered":"2024-09-10T22:58:34.185664+0000","last_clean":"2024-09-10T22:58:34.185664+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:58:34.185664+0000","last_undegraded":"2024-09-10T22:58:34.185664+0000","last_fullsized":"2024-09-10T22:58:34.185664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464494+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:58:34.464494+0000","last_peered":"2024-09-10T22:58:34.464494+0000","last_clean":"2024-09-10T22:58:34.464494+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:58:34.464494+0000","last_undegraded":"2024-09-10T22:58:34.464494+0000","last_fullsized":"2024-09-10T22:58:34.464494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187941+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:58:34.187941+0000","last_peered":"2024-09-10T22:58:34.187941+0000","last_clean":"2024-09-10T22:58:34.187941+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:58:34.187941+0000","last_undegraded":"2024-09-10T22:58:34.187941+0000","last_fullsized":"2024-09-10T22:58:34.187941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462839+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:58:34.462839+0000","last_peered":"2024-09-10T22:58:34.462839+0000","last_clean":"2024-09-10T22:58:34.462839+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:58:34.462839+0000","last_undegraded":"2024-09-10T22:58:34.462839+0000","last_fullsized":"2024-09-10T22:58:34.462839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188745+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:58:34.188745+0000","last_peered":"2024-09-10T22:58:34.188745+0000","last_clean":"2024-09-10T22:58:34.188745+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:58:34.188745+0000","last_undegraded":"2024-09-10T22:58:34.188745+0000","last_fullsized":"2024-09-10T22:58:34.188745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187786+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:58:34.187786+0000","last_peered":"2024-09-10T22:58:34.187786+0000","last_clean":"2024-09-10T22:58:34.187786+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:58:34.187786+0000","last_undegraded":"2024-09-10T22:58:34.187786+0000","last_fullsized":"2024-09-10T22:58:34.187786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186083+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:58:34.186083+0000","last_peered":"2024-09-10T22:58:34.186083+0000","last_clean":"2024-09-10T22:58:34.186083+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:58:34.186083+0000","last_undegraded":"2024-09-10T22:58:34.186083+0000","last_fullsized":"2024-09-10T22:58:34.186083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462767+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:58:34.462767+0000","last_peered":"2024-09-10T22:58:34.462767+0000","last_clean":"2024-09-10T22:58:34.462767+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:58:34.462767+0000","last_undegraded":"2024-09-10T22:58:34.462767+0000","last_fullsized":"2024-09-10T22:58:34.462767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184820+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:58:34.184820+0000","last_peered":"2024-09-10T22:58:34.184820+0000","last_clean":"2024-09-10T22:58:34.184820+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:58:34.184820+0000","last_undegraded":"2024-09-10T22:58:34.184820+0000","last_fullsized":"2024-09-10T22:58:34.184820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464201+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:58:34.464201+0000","last_peered":"2024-09-10T22:58:34.464201+0000","last_clean":"2024-09-10T22:58:34.464201+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:58:34.464201+0000","last_undegraded":"2024-09-10T22:58:34.464201+0000","last_fullsized":"2024-09-10T22:58:34.464201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587613+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:58:34.587613+0000","last_peered":"2024-09-10T22:58:34.587613+0000","last_clean":"2024-09-10T22:58:34.587613+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:58:34.587613+0000","last_undegraded":"2024-09-10T22:58:34.587613+0000","last_fullsized":"2024-09-10T22:58:34.587613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466740+0000","last_change":"2024-09-10T22:58:34.466034+0000","last_active":"2024-09-10T22:58:34.466740+0000","last_peered":"2024-09-10T22:58:34.466740+0000","last_clean":"2024-09-10T22:58:34.466740+0000","last_became_active":"2024-09-10T22:58:34.465493+0000","last_became_peered":"2024-09-10T22:58:34.465493+0000","last_unstale":"2024-09-10T22:58:34.466740+0000","last_undegraded":"2024-09-10T22:58:34.466740+0000","last_fullsized":"2024-09-10T22:58:34.466740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:43.360753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666497+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:58:34.666497+0000","last_peered":"2024-09-10T22:58:34.666497+0000","last_clean":"2024-09-10T22:58:34.666497+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:58:34.666497+0000","last_undegraded":"2024-09-10T22:58:34.666497+0000","last_fullsized":"2024-09-10T22:58:34.666497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188622+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:58:34.188622+0000","last_peered":"2024-09-10T22:58:34.188622+0000","last_clean":"2024-09-10T22:58:34.188622+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:58:34.188622+0000","last_undegraded":"2024-09-10T22:58:34.188622+0000","last_fullsized":"2024-09-10T22:58:34.188622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188981+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:58:34.188981+0000","last_peered":"2024-09-10T22:58:34.188981+0000","last_clean":"2024-09-10T22:58:34.188981+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:58:34.188981+0000","last_undegraded":"2024-09-10T22:58:34.188981+0000","last_fullsized":"2024-09-10T22:58:34.188981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463075+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:58:34.463075+0000","last_peered":"2024-09-10T22:58:34.463075+0000","last_clean":"2024-09-10T22:58:34.463075+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:58:34.463075+0000","last_undegraded":"2024-09-10T22:58:34.463075+0000","last_fullsized":"2024-09-10T22:58:34.463075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'97","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:26.296239+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:59:26.296239+0000","last_peered":"2024-09-10T22:59:26.296239+0000","last_clean":"2024-09-10T22:59:26.296239+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:59:26.296239+0000","last_undegraded":"2024-09-10T22:59:26.296239+0000","last_fullsized":"2024-09-10T22:59:26.296239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462341+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:58:34.462341+0000","last_peered":"2024-09-10T22:58:34.462341+0000","last_clean":"2024-09-10T22:58:34.462341+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:58:34.462341+0000","last_undegraded":"2024-09-10T22:58:34.462341+0000","last_fullsized":"2024-09-10T22:58:34.462341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186602+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:58:34.186602+0000","last_peered":"2024-09-10T22:58:34.186602+0000","last_clean":"2024-09-10T22:58:34.186602+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:58:34.186602+0000","last_undegraded":"2024-09-10T22:58:34.186602+0000","last_fullsized":"2024-09-10T22:58:34.186602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464970+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:58:34.464970+0000","last_peered":"2024-09-10T22:58:34.464970+0000","last_clean":"2024-09-10T22:58:34.464970+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:58:34.464970+0000","last_undegraded":"2024-09-10T22:58:34.464970+0000","last_fullsized":"2024-09-10T22:58:34.464970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185884+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:58:34.185884+0000","last_peered":"2024-09-10T22:58:34.185884+0000","last_clean":"2024-09-10T22:58:34.185884+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:58:34.185884+0000","last_undegraded":"2024-09-10T22:58:34.185884+0000","last_fullsized":"2024-09-10T22:58:34.185884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189026+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:58:34.189026+0000","last_peered":"2024-09-10T22:58:34.189026+0000","last_clean":"2024-09-10T22:58:34.189026+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:58:34.189026+0000","last_undegraded":"2024-09-10T22:58:34.189026+0000","last_fullsized":"2024-09-10T22:58:34.189026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184648+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:58:34.184648+0000","last_peered":"2024-09-10T22:58:34.184648+0000","last_clean":"2024-09-10T22:58:34.184648+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:58:34.184648+0000","last_undegraded":"2024-09-10T22:58:34.184648+0000","last_fullsized":"2024-09-10T22:58:34.184648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464415+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:58:34.464415+0000","last_peered":"2024-09-10T22:58:34.464415+0000","last_clean":"2024-09-10T22:58:34.464415+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:58:34.464415+0000","last_undegraded":"2024-09-10T22:58:34.464415+0000","last_fullsized":"2024-09-10T22:58:34.464415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188871+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:58:34.188871+0000","last_peered":"2024-09-10T22:58:34.188871+0000","last_clean":"2024-09-10T22:58:34.188871+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:58:34.188871+0000","last_undegraded":"2024-09-10T22:58:34.188871+0000","last_fullsized":"2024-09-10T22:58:34.188871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185784+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:58:34.185784+0000","last_peered":"2024-09-10T22:58:34.185784+0000","last_clean":"2024-09-10T22:58:34.185784+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:58:34.185784+0000","last_undegraded":"2024-09-10T22:58:34.185784+0000","last_fullsized":"2024-09-10T22:58:34.185784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186540+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:58:34.186540+0000","last_peered":"2024-09-10T22:58:34.186540+0000","last_clean":"2024-09-10T22:58:34.186540+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:58:34.186540+0000","last_undegraded":"2024-09-10T22:58:34.186540+0000","last_fullsized":"2024-09-10T22:58:34.186540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602469+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:58:34.602469+0000","last_peered":"2024-09-10T22:58:34.602469+0000","last_clean":"2024-09-10T22:58:34.602469+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:58:34.602469+0000","last_undegraded":"2024-09-10T22:58:34.602469+0000","last_fullsized":"2024-09-10T22:58:34.602469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:23.694786+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:59:23.694786+0000","last_peered":"2024-09-10T22:59:23.694786+0000","last_clean":"2024-09-10T22:59:23.694786+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:59:23.694786+0000","last_undegraded":"2024-09-10T22:59:23.694786+0000","last_fullsized":"2024-09-10T22:59:23.694786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186783+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:58:34.186783+0000","last_peered":"2024-09-10T22:58:34.186783+0000","last_clean":"2024-09-10T22:58:34.186783+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:58:34.186783+0000","last_undegraded":"2024-09-10T22:58:34.186783+0000","last_fullsized":"2024-09-10T22:58:34.186783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.190954+0000","last_change":"2024-09-10T22:58:34.190886+0000","last_active":"2024-09-10T22:58:34.190954+0000","last_peered":"2024-09-10T22:58:34.190954+0000","last_clean":"2024-09-10T22:58:34.190954+0000","last_became_active":"2024-09-10T22:58:34.190534+0000","last_became_peered":"2024-09-10T22:58:34.190534+0000","last_unstale":"2024-09-10T22:58:34.190954+0000","last_undegraded":"2024-09-10T22:58:34.190954+0000","last_fullsized":"2024-09-10T22:58:34.190954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:07.888931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185571+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:58:34.185571+0000","last_peered":"2024-09-10T22:58:34.185571+0000","last_clean":"2024-09-10T22:58:34.185571+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:58:34.185571+0000","last_undegraded":"2024-09-10T22:58:34.185571+0000","last_fullsized":"2024-09-10T22:58:34.185571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.603153+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:58:34.603153+0000","last_peered":"2024-09-10T22:58:34.603153+0000","last_clean":"2024-09-10T22:58:34.603153+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:58:34.603153+0000","last_undegraded":"2024-09-10T22:58:34.603153+0000","last_fullsized":"2024-09-10T22:58:34.603153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184347+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:58:34.184347+0000","last_peered":"2024-09-10T22:58:34.184347+0000","last_clean":"2024-09-10T22:58:34.184347+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:58:34.184347+0000","last_undegraded":"2024-09-10T22:58:34.184347+0000","last_fullsized":"2024-09-10T22:58:34.184347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188099+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:58:34.188099+0000","last_peered":"2024-09-10T22:58:34.188099+0000","last_clean":"2024-09-10T22:58:34.188099+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:58:34.188099+0000","last_undegraded":"2024-09-10T22:58:34.188099+0000","last_fullsized":"2024-09-10T22:58:34.188099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185870+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:58:34.185870+0000","last_peered":"2024-09-10T22:58:34.185870+0000","last_clean":"2024-09-10T22:58:34.185870+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:58:34.185870+0000","last_undegraded":"2024-09-10T22:58:34.185870+0000","last_fullsized":"2024-09-10T22:58:34.185870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186444+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:58:34.186444+0000","last_peered":"2024-09-10T22:58:34.186444+0000","last_clean":"2024-09-10T22:58:34.186444+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:58:34.186444+0000","last_undegraded":"2024-09-10T22:58:34.186444+0000","last_fullsized":"2024-09-10T22:58:34.186444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186412+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:58:34.186412+0000","last_peered":"2024-09-10T22:58:34.186412+0000","last_clean":"2024-09-10T22:58:34.186412+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:58:34.186412+0000","last_undegraded":"2024-09-10T22:58:34.186412+0000","last_fullsized":"2024-09-10T22:58:34.186412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189203+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:58:34.189203+0000","last_peered":"2024-09-10T22:58:34.189203+0000","last_clean":"2024-09-10T22:58:34.189203+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:58:34.189203+0000","last_undegraded":"2024-09-10T22:58:34.189203+0000","last_fullsized":"2024-09-10T22:58:34.189203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587205+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:58:34.587205+0000","last_peered":"2024-09-10T22:58:34.587205+0000","last_clean":"2024-09-10T22:58:34.587205+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:58:34.587205+0000","last_undegraded":"2024-09-10T22:58:34.587205+0000","last_fullsized":"2024-09-10T22:58:34.587205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185988+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:58:34.185988+0000","last_peered":"2024-09-10T22:58:34.185988+0000","last_clean":"2024-09-10T22:58:34.185988+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:58:34.185988+0000","last_undegraded":"2024-09-10T22:58:34.185988+0000","last_fullsized":"2024-09-10T22:58:34.185988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602862+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:58:34.602862+0000","last_peered":"2024-09-10T22:58:34.602862+0000","last_clean":"2024-09-10T22:58:34.602862+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:58:34.602862+0000","last_undegraded":"2024-09-10T22:58:34.602862+0000","last_fullsized":"2024-09-10T22:58:34.602862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666303+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:58:34.666303+0000","last_peered":"2024-09-10T22:58:34.666303+0000","last_clean":"2024-09-10T22:58:34.666303+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:58:34.666303+0000","last_undegraded":"2024-09-10T22:58:34.666303+0000","last_fullsized":"2024-09-10T22:58:34.666303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186147+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:58:34.186147+0000","last_peered":"2024-09-10T22:58:34.186147+0000","last_clean":"2024-09-10T22:58:34.186147+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:58:34.186147+0000","last_undegraded":"2024-09-10T22:58:34.186147+0000","last_fullsized":"2024-09-10T22:58:34.186147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187257+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:58:34.187257+0000","last_peered":"2024-09-10T22:58:34.187257+0000","last_clean":"2024-09-10T22:58:34.187257+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:58:34.187257+0000","last_undegraded":"2024-09-10T22:58:34.187257+0000","last_fullsized":"2024-09-10T22:58:34.187257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.601976+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:58:34.601976+0000","last_peered":"2024-09-10T22:58:34.601976+0000","last_clean":"2024-09-10T22:58:34.601976+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:58:34.601976+0000","last_undegraded":"2024-09-10T22:58:34.601976+0000","last_fullsized":"2024-09-10T22:58:34.601976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186276+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:58:34.186276+0000","last_peered":"2024-09-10T22:58:34.186276+0000","last_clean":"2024-09-10T22:58:34.186276+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:58:34.186276+0000","last_undegraded":"2024-09-10T22:58:34.186276+0000","last_fullsized":"2024-09-10T22:58:34.186276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465088+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:58:34.465088+0000","last_peered":"2024-09-10T22:58:34.465088+0000","last_clean":"2024-09-10T22:58:34.465088+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:58:34.465088+0000","last_undegraded":"2024-09-10T22:58:34.465088+0000","last_fullsized":"2024-09-10T22:58:34.465088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602472+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:58:34.602472+0000","last_peered":"2024-09-10T22:58:34.602472+0000","last_clean":"2024-09-10T22:58:34.602472+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:58:34.602472+0000","last_undegraded":"2024-09-10T22:58:34.602472+0000","last_fullsized":"2024-09-10T22:58:34.602472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602600+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:58:34.602600+0000","last_peered":"2024-09-10T22:58:34.602600+0000","last_clean":"2024-09-10T22:58:34.602600+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:58:34.602600+0000","last_undegraded":"2024-09-10T22:58:34.602600+0000","last_fullsized":"2024-09-10T22:58:34.602600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185766+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:58:34.185766+0000","last_peered":"2024-09-10T22:58:34.185766+0000","last_clean":"2024-09-10T22:58:34.185766+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:58:34.185766+0000","last_undegraded":"2024-09-10T22:58:34.185766+0000","last_fullsized":"2024-09-10T22:58:34.185766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466980+0000","last_change":"2024-09-10T22:58:34.466291+0000","last_active":"2024-09-10T22:58:34.466980+0000","last_peered":"2024-09-10T22:58:34.466980+0000","last_clean":"2024-09-10T22:58:34.466980+0000","last_became_active":"2024-09-10T22:58:34.465710+0000","last_became_peered":"2024-09-10T22:58:34.465710+0000","last_unstale":"2024-09-10T22:58:34.466980+0000","last_undegraded":"2024-09-10T22:58:34.466980+0000","last_fullsized":"2024-09-10T22:58:34.466980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:32.445364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185427+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:58:34.185427+0000","last_peered":"2024-09-10T22:58:34.185427+0000","last_clean":"2024-09-10T22:58:34.185427+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:58:34.185427+0000","last_undegraded":"2024-09-10T22:58:34.185427+0000","last_fullsized":"2024-09-10T22:58:34.185427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:23.695454+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:59:23.695454+0000","last_peered":"2024-09-10T22:59:23.695454+0000","last_clean":"2024-09-10T22:59:23.695454+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:59:23.695454+0000","last_undegraded":"2024-09-10T22:59:23.695454+0000","last_fullsized":"2024-09-10T22:59:23.695454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184894+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:58:34.184894+0000","last_peered":"2024-09-10T22:58:34.184894+0000","last_clean":"2024-09-10T22:58:34.184894+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:58:34.184894+0000","last_undegraded":"2024-09-10T22:58:34.184894+0000","last_fullsized":"2024-09-10T22:58:34.184894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185942+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:58:34.185942+0000","last_peered":"2024-09-10T22:58:34.185942+0000","last_clean":"2024-09-10T22:58:34.185942+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:58:34.185942+0000","last_undegraded":"2024-09-10T22:58:34.185942+0000","last_fullsized":"2024-09-10T22:58:34.185942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602689+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:58:34.602689+0000","last_peered":"2024-09-10T22:58:34.602689+0000","last_clean":"2024-09-10T22:58:34.602689+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:58:34.602689+0000","last_undegraded":"2024-09-10T22:58:34.602689+0000","last_fullsized":"2024-09-10T22:58:34.602689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185960+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:58:34.185960+0000","last_peered":"2024-09-10T22:58:34.185960+0000","last_clean":"2024-09-10T22:58:34.185960+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:58:34.185960+0000","last_undegraded":"2024-09-10T22:58:34.185960+0000","last_fullsized":"2024-09-10T22:58:34.185960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186578+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:58:34.186578+0000","last_peered":"2024-09-10T22:58:34.186578+0000","last_clean":"2024-09-10T22:58:34.186578+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:58:34.186578+0000","last_undegraded":"2024-09-10T22:58:34.186578+0000","last_fullsized":"2024-09-10T22:58:34.186578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188921+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:58:34.188921+0000","last_peered":"2024-09-10T22:58:34.188921+0000","last_clean":"2024-09-10T22:58:34.188921+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:58:34.188921+0000","last_undegraded":"2024-09-10T22:58:34.188921+0000","last_fullsized":"2024-09-10T22:58:34.188921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602556+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:58:34.602556+0000","last_peered":"2024-09-10T22:58:34.602556+0000","last_clean":"2024-09-10T22:58:34.602556+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:58:34.602556+0000","last_undegraded":"2024-09-10T22:58:34.602556+0000","last_fullsized":"2024-09-10T22:58:34.602556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464901+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:58:34.464901+0000","last_peered":"2024-09-10T22:58:34.464901+0000","last_clean":"2024-09-10T22:58:34.464901+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:58:34.464901+0000","last_undegraded":"2024-09-10T22:58:34.464901+0000","last_fullsized":"2024-09-10T22:58:34.464901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:25.700231+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:59:25.700231+0000","last_peered":"2024-09-10T22:59:25.700231+0000","last_clean":"2024-09-10T22:59:25.700231+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:59:25.700231+0000","last_undegraded":"2024-09-10T22:59:25.700231+0000","last_fullsized":"2024-09-10T22:59:25.700231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62814,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":277,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.604692+0000","last_change":"2024-09-10T22:58:34.604157+0000","last_active":"2024-09-10T22:58:34.604692+0000","last_peered":"2024-09-10T22:58:34.604692+0000","last_clean":"2024-09-10T22:58:34.604692+0000","last_became_active":"2024-09-10T22:58:34.603682+0000","last_became_peered":"2024-09-10T22:58:34.603682+0000","last_unstale":"2024-09-10T22:58:34.604692+0000","last_undegraded":"2024-09-10T22:58:34.604692+0000","last_fullsized":"2024-09-10T22:58:34.604692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:44.610556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587420+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:58:34.587420+0000","last_peered":"2024-09-10T22:58:34.587420+0000","last_clean":"2024-09-10T22:58:34.587420+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:58:34.587420+0000","last_undegraded":"2024-09-10T22:58:34.587420+0000","last_fullsized":"2024-09-10T22:58:34.587420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602989+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:58:34.602989+0000","last_peered":"2024-09-10T22:58:34.602989+0000","last_clean":"2024-09-10T22:58:34.602989+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:58:34.602989+0000","last_undegraded":"2024-09-10T22:58:34.602989+0000","last_fullsized":"2024-09-10T22:58:34.602989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.589489+0000","last_change":"2024-09-10T22:58:34.588867+0000","last_active":"2024-09-10T22:58:34.589489+0000","last_peered":"2024-09-10T22:58:34.589489+0000","last_clean":"2024-09-10T22:58:34.589489+0000","last_became_active":"2024-09-10T22:58:34.588410+0000","last_became_peered":"2024-09-10T22:58:34.588410+0000","last_unstale":"2024-09-10T22:58:34.589489+0000","last_undegraded":"2024-09-10T22:58:34.589489+0000","last_fullsized":"2024-09-10T22:58:34.589489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:24.765755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666891+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:58:34.666891+0000","last_peered":"2024-09-10T22:58:34.666891+0000","last_clean":"2024-09-10T22:58:34.666891+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:58:34.666891+0000","last_undegraded":"2024-09-10T22:58:34.666891+0000","last_fullsized":"2024-09-10T22:58:34.666891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185860+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:58:34.185860+0000","last_peered":"2024-09-10T22:58:34.185860+0000","last_clean":"2024-09-10T22:58:34.185860+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:58:34.185860+0000","last_undegraded":"2024-09-10T22:58:34.185860+0000","last_fullsized":"2024-09-10T22:58:34.185860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184137+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:58:34.184137+0000","last_peered":"2024-09-10T22:58:34.184137+0000","last_clean":"2024-09-10T22:58:34.184137+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:58:34.184137+0000","last_undegraded":"2024-09-10T22:58:34.184137+0000","last_fullsized":"2024-09-10T22:58:34.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:24.698215+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:59:24.698215+0000","last_peered":"2024-09-10T22:59:24.698215+0000","last_clean":"2024-09-10T22:59:24.698215+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:59:24.698215+0000","last_undegraded":"2024-09-10T22:59:24.698215+0000","last_fullsized":"2024-09-10T22:59:24.698215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462520+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:58:34.462520+0000","last_peered":"2024-09-10T22:58:34.462520+0000","last_clean":"2024-09-10T22:58:34.462520+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:58:34.462520+0000","last_undegraded":"2024-09-10T22:58:34.462520+0000","last_fullsized":"2024-09-10T22:58:34.462520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666395+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:58:34.666395+0000","last_peered":"2024-09-10T22:58:34.666395+0000","last_clean":"2024-09-10T22:58:34.666395+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:58:34.666395+0000","last_undegraded":"2024-09-10T22:58:34.666395+0000","last_fullsized":"2024-09-10T22:58:34.666395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184232+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:58:34.184232+0000","last_peered":"2024-09-10T22:58:34.184232+0000","last_clean":"2024-09-10T22:58:34.184232+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:58:34.184232+0000","last_undegraded":"2024-09-10T22:58:34.184232+0000","last_fullsized":"2024-09-10T22:58:34.184232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602683+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:58:34.602683+0000","last_peered":"2024-09-10T22:58:34.602683+0000","last_clean":"2024-09-10T22:58:34.602683+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:58:34.602683+0000","last_undegraded":"2024-09-10T22:58:34.602683+0000","last_fullsized":"2024-09-10T22:58:34.602683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186357+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:58:34.186357+0000","last_peered":"2024-09-10T22:58:34.186357+0000","last_clean":"2024-09-10T22:58:34.186357+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:58:34.186357+0000","last_undegraded":"2024-09-10T22:58:34.186357+0000","last_fullsized":"2024-09-10T22:58:34.186357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462533+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:58:34.462533+0000","last_peered":"2024-09-10T22:58:34.462533+0000","last_clean":"2024-09-10T22:58:34.462533+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:58:34.462533+0000","last_undegraded":"2024-09-10T22:58:34.462533+0000","last_fullsized":"2024-09-10T22:58:34.462533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186324+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:58:34.186324+0000","last_peered":"2024-09-10T22:58:34.186324+0000","last_clean":"2024-09-10T22:58:34.186324+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:58:34.186324+0000","last_undegraded":"2024-09-10T22:58:34.186324+0000","last_fullsized":"2024-09-10T22:58:34.186324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189561+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:58:34.189561+0000","last_peered":"2024-09-10T22:58:34.189561+0000","last_clean":"2024-09-10T22:58:34.189561+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:58:34.189561+0000","last_undegraded":"2024-09-10T22:58:34.189561+0000","last_fullsized":"2024-09-10T22:58:34.189561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602398+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:58:34.602398+0000","last_peered":"2024-09-10T22:58:34.602398+0000","last_clean":"2024-09-10T22:58:34.602398+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:58:34.602398+0000","last_undegraded":"2024-09-10T22:58:34.602398+0000","last_fullsized":"2024-09-10T22:58:34.602398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188728+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:58:34.188728+0000","last_peered":"2024-09-10T22:58:34.188728+0000","last_clean":"2024-09-10T22:58:34.188728+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:58:34.188728+0000","last_undegraded":"2024-09-10T22:58:34.188728+0000","last_fullsized":"2024-09-10T22:58:34.188728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:22.694095+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:59:22.694095+0000","last_peered":"2024-09-10T22:59:22.694095+0000","last_clean":"2024-09-10T22:59:22.694095+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:59:22.694095+0000","last_undegraded":"2024-09-10T22:59:22.694095+0000","last_fullsized":"2024-09-10T22:59:22.694095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464546+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:58:34.464546+0000","last_peered":"2024-09-10T22:58:34.464546+0000","last_clean":"2024-09-10T22:58:34.464546+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:58:34.464546+0000","last_undegraded":"2024-09-10T22:58:34.464546+0000","last_fullsized":"2024-09-10T22:58:34.464546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189292+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:58:34.189292+0000","last_peered":"2024-09-10T22:58:34.189292+0000","last_clean":"2024-09-10T22:58:34.189292+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:58:34.189292+0000","last_undegraded":"2024-09-10T22:58:34.189292+0000","last_fullsized":"2024-09-10T22:58:34.189292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185970+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:58:34.185970+0000","last_peered":"2024-09-10T22:58:34.185970+0000","last_clean":"2024-09-10T22:58:34.185970+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:58:34.185970+0000","last_undegraded":"2024-09-10T22:58:34.185970+0000","last_fullsized":"2024-09-10T22:58:34.185970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189242+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:58:34.189242+0000","last_peered":"2024-09-10T22:58:34.189242+0000","last_clean":"2024-09-10T22:58:34.189242+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:58:34.189242+0000","last_undegraded":"2024-09-10T22:58:34.189242+0000","last_fullsized":"2024-09-10T22:58:34.189242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188923+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:58:34.188923+0000","last_peered":"2024-09-10T22:58:34.188923+0000","last_clean":"2024-09-10T22:58:34.188923+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:58:34.188923+0000","last_undegraded":"2024-09-10T22:58:34.188923+0000","last_fullsized":"2024-09-10T22:58:34.188923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666291+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:58:34.666291+0000","last_peered":"2024-09-10T22:58:34.666291+0000","last_clean":"2024-09-10T22:58:34.666291+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:58:34.666291+0000","last_undegraded":"2024-09-10T22:58:34.666291+0000","last_fullsized":"2024-09-10T22:58:34.666291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186557+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:58:34.186557+0000","last_peered":"2024-09-10T22:58:34.186557+0000","last_clean":"2024-09-10T22:58:34.186557+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:58:34.186557+0000","last_undegraded":"2024-09-10T22:58:34.186557+0000","last_fullsized":"2024-09-10T22:58:34.186557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186283+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:58:34.186283+0000","last_peered":"2024-09-10T22:58:34.186283+0000","last_clean":"2024-09-10T22:58:34.186283+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:58:34.186283+0000","last_undegraded":"2024-09-10T22:58:34.186283+0000","last_fullsized":"2024-09-10T22:58:34.186283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189258+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:58:34.189258+0000","last_peered":"2024-09-10T22:58:34.189258+0000","last_clean":"2024-09-10T22:58:34.189258+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:58:34.189258+0000","last_undegraded":"2024-09-10T22:58:34.189258+0000","last_fullsized":"2024-09-10T22:58:34.189258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462967+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:58:34.462967+0000","last_peered":"2024-09-10T22:58:34.462967+0000","last_clean":"2024-09-10T22:58:34.462967+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:58:34.462967+0000","last_undegraded":"2024-09-10T22:58:34.462967+0000","last_fullsized":"2024-09-10T22:58:34.462967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602043+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:58:34.602043+0000","last_peered":"2024-09-10T22:58:34.602043+0000","last_clean":"2024-09-10T22:58:34.602043+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:58:34.602043+0000","last_undegraded":"2024-09-10T22:58:34.602043+0000","last_fullsized":"2024-09-10T22:58:34.602043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462583+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:58:34.462583+0000","last_peered":"2024-09-10T22:58:34.462583+0000","last_clean":"2024-09-10T22:58:34.462583+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:58:34.462583+0000","last_undegraded":"2024-09-10T22:58:34.462583+0000","last_fullsized":"2024-09-10T22:58:34.462583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186971+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:58:34.186971+0000","last_peered":"2024-09-10T22:58:34.186971+0000","last_clean":"2024-09-10T22:58:34.186971+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:58:34.186971+0000","last_undegraded":"2024-09-10T22:58:34.186971+0000","last_fullsized":"2024-09-10T22:58:34.186971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186021+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:58:34.186021+0000","last_peered":"2024-09-10T22:58:34.186021+0000","last_clean":"2024-09-10T22:58:34.186021+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:58:34.186021+0000","last_undegraded":"2024-09-10T22:58:34.186021+0000","last_fullsized":"2024-09-10T22:58:34.186021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189081+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:58:34.189081+0000","last_peered":"2024-09-10T22:58:34.189081+0000","last_clean":"2024-09-10T22:58:34.189081+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:58:34.189081+0000","last_undegraded":"2024-09-10T22:58:34.189081+0000","last_fullsized":"2024-09-10T22:58:34.189081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184788+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:58:34.184788+0000","last_peered":"2024-09-10T22:58:34.184788+0000","last_clean":"2024-09-10T22:58:34.184788+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:58:34.184788+0000","last_undegraded":"2024-09-10T22:58:34.184788+0000","last_fullsized":"2024-09-10T22:58:34.184788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186656+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:58:34.186656+0000","last_peered":"2024-09-10T22:58:34.186656+0000","last_clean":"2024-09-10T22:58:34.186656+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:58:34.186656+0000","last_undegraded":"2024-09-10T22:58:34.186656+0000","last_fullsized":"2024-09-10T22:58:34.186656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587130+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:58:34.587130+0000","last_peered":"2024-09-10T22:58:34.587130+0000","last_clean":"2024-09-10T22:58:34.587130+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:58:34.587130+0000","last_undegraded":"2024-09-10T22:58:34.587130+0000","last_fullsized":"2024-09-10T22:58:34.587130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666125+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:58:34.666125+0000","last_peered":"2024-09-10T22:58:34.666125+0000","last_clean":"2024-09-10T22:58:34.666125+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:58:34.666125+0000","last_undegraded":"2024-09-10T22:58:34.666125+0000","last_fullsized":"2024-09-10T22:58:34.666125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666081+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:58:34.666081+0000","last_peered":"2024-09-10T22:58:34.666081+0000","last_clean":"2024-09-10T22:58:34.666081+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:58:34.666081+0000","last_undegraded":"2024-09-10T22:58:34.666081+0000","last_fullsized":"2024-09-10T22:58:34.666081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186817+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:58:34.186817+0000","last_peered":"2024-09-10T22:58:34.186817+0000","last_clean":"2024-09-10T22:58:34.186817+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:58:34.186817+0000","last_undegraded":"2024-09-10T22:58:34.186817+0000","last_fullsized":"2024-09-10T22:58:34.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188049+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:58:34.188049+0000","last_peered":"2024-09-10T22:58:34.188049+0000","last_clean":"2024-09-10T22:58:34.188049+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:58:34.188049+0000","last_undegraded":"2024-09-10T22:58:34.188049+0000","last_fullsized":"2024-09-10T22:58:34.188049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186351+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:58:34.186351+0000","last_peered":"2024-09-10T22:58:34.186351+0000","last_clean":"2024-09-10T22:58:34.186351+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:58:34.186351+0000","last_undegraded":"2024-09-10T22:58:34.186351+0000","last_fullsized":"2024-09-10T22:58:34.186351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189733+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:58:34.189733+0000","last_peered":"2024-09-10T22:58:34.189733+0000","last_clean":"2024-09-10T22:58:34.189733+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:58:34.189733+0000","last_undegraded":"2024-09-10T22:58:34.189733+0000","last_fullsized":"2024-09-10T22:58:34.189733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587752+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:58:34.587752+0000","last_peered":"2024-09-10T22:58:34.587752+0000","last_clean":"2024-09-10T22:58:34.587752+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:58:34.587752+0000","last_undegraded":"2024-09-10T22:58:34.587752+0000","last_fullsized":"2024-09-10T22:58:34.587752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462478+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:58:34.462478+0000","last_peered":"2024-09-10T22:58:34.462478+0000","last_clean":"2024-09-10T22:58:34.462478+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:58:34.462478+0000","last_undegraded":"2024-09-10T22:58:34.462478+0000","last_fullsized":"2024-09-10T22:58:34.462478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185517+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:58:34.185517+0000","last_peered":"2024-09-10T22:58:34.185517+0000","last_clean":"2024-09-10T22:58:34.185517+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:58:34.185517+0000","last_undegraded":"2024-09-10T22:58:34.185517+0000","last_fullsized":"2024-09-10T22:58:34.185517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189448+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:58:34.189448+0000","last_peered":"2024-09-10T22:58:34.189448+0000","last_clean":"2024-09-10T22:58:34.189448+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:58:34.189448+0000","last_undegraded":"2024-09-10T22:58:34.189448+0000","last_fullsized":"2024-09-10T22:58:34.189448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464386+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:58:34.464386+0000","last_peered":"2024-09-10T22:58:34.464386+0000","last_clean":"2024-09-10T22:58:34.464386+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:58:34.464386+0000","last_undegraded":"2024-09-10T22:58:34.464386+0000","last_fullsized":"2024-09-10T22:58:34.464386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187920+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:58:34.187920+0000","last_peered":"2024-09-10T22:58:34.187920+0000","last_clean":"2024-09-10T22:58:34.187920+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:58:34.187920+0000","last_undegraded":"2024-09-10T22:58:34.187920+0000","last_fullsized":"2024-09-10T22:58:34.187920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666406+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:58:34.666406+0000","last_peered":"2024-09-10T22:58:34.666406+0000","last_clean":"2024-09-10T22:58:34.666406+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:58:34.666406+0000","last_undegraded":"2024-09-10T22:58:34.666406+0000","last_fullsized":"2024-09-10T22:58:34.666406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666421+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:58:34.666421+0000","last_peered":"2024-09-10T22:58:34.666421+0000","last_clean":"2024-09-10T22:58:34.666421+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:58:34.666421+0000","last_undegraded":"2024-09-10T22:58:34.666421+0000","last_fullsized":"2024-09-10T22:58:34.666421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186517+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:58:34.186517+0000","last_peered":"2024-09-10T22:58:34.186517+0000","last_clean":"2024-09-10T22:58:34.186517+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:58:34.186517+0000","last_undegraded":"2024-09-10T22:58:34.186517+0000","last_fullsized":"2024-09-10T22:58:34.186517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":461158,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1125,"num_read_kb":755,"num_write":339,"num_write_kb":1346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":1384034,"data_compressed":56652,"data_compressed_allocated":73728,"data_compressed_original":1155072,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612662,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518482,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808891,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1179648,"data_stored":1562721,"data_compressed":83571,"data_compressed_allocated":110592,"data_compressed_original":1040384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":915918,"data_compressed":48912,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518414,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397580,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1115562,"data_compressed":65480,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561107,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":513782,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1110657,"data_compressed":65480,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":519037,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117092,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":513976,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":577122,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":577167,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":999424,"data_stored":515170,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19272,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":458661,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:59:30.617 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:59:30.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:59:30.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:30 smithi145 ceph-mon[34397]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:30.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:30 smithi067 ceph-mon[30939]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:30.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:30 smithi092 ceph-mon[34223]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:31.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:31 smithi067 ceph-mon[30939]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:31 smithi145 ceph-mon[34397]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 -- 172.21.15.67:0/2041445234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b201087b0 msgr2=0x7f5b20108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 --2- 172.21.15.67:0/2041445234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b201087b0 0x7f5b20108bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009900 tx=0x7f5b0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 -- 172.21.15.67:0/2041445234 shutdown_connections 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 --2- 172.21.15.67:0/2041445234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20109130 0x7f5b2010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 --2- 172.21.15.67:0/2041445234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b201087b0 0x7f5b20108bf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 --2- 172.21.15.67:0/2041445234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b20102da0 0x7f5b20103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 -- 172.21.15.67:0/2041445234 >> 172.21.15.67:0/2041445234 conn(0x7f5b200fe810 msgr2=0x7f5b20100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 -- 172.21.15.67:0/2041445234 shutdown_connections 2024-09-10T22:59:31.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.657+0000 7f5b27200640 1 -- 172.21.15.67:0/2041445234 wait complete. 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.658+0000 7f5b27200640 1 Processor -- start 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.658+0000 7f5b27200640 1 -- start start 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20102da0 0x7f5b2019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b201087b0 0x7f5b2019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20112cf0 con 0x7f5b20109130 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20112b70 con 0x7f5b20102da0 2024-09-10T22:59:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b27200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20112e70 con 0x7f5b201087b0 2024-09-10T22:59:31.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44562/0 (socket says 172.21.15.67:44562) 2024-09-10T22:59:31.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b25200640 1 -- 172.21.15.67:0/684503923 learned_addr learned my addr 172.21.15.67:0/684503923 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:31.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b1fe00640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20102da0 0x7f5b2019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.659+0000 7f5b1f400640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b201087b0 0x7f5b2019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 -- 172.21.15.67:0/684503923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b201087b0 msgr2=0x7f5b2019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b201087b0 0x7f5b2019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 -- 172.21.15.67:0/684503923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20102da0 msgr2=0x7f5b2019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20102da0 0x7f5b2019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 -- 172.21.15.67:0/684503923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b201a13e0 con 0x7f5b20109130 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b25200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800cce0 tx=0x7f5b1800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800f450 con 0x7f5b20109130 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b1800fa80 con 0x7f5b20109130 2024-09-10T22:59:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b201a16d0 con 0x7f5b20109130 2024-09-10T22:59:31.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18018ad0 con 0x7f5b20109130 2024-09-10T22:59:31.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.660+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b201a7d90 con 0x7f5b20109130 2024-09-10T22:59:31.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.662+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b20103180 con 0x7f5b20109130 2024-09-10T22:59:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.665+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f5b1800f5f0 con 0x7f5b20109130 2024-09-10T22:59:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.666+0000 7f5b1cc00640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 0x7f5b040b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.666+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f5b180e6ac0 con 0x7f5b20109130 2024-09-10T22:59:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.666+0000 7f5b1fe00640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 0x7f5b040b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.667+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5b18014030 con 0x7f5b20109130 2024-09-10T22:59:31.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.667+0000 7f5b1fe00640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 0x7f5b040b7900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005fd0 tx=0x7f5b1000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:31 smithi092 ceph-mon[34223]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:31.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.858+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b20063590 con 0x7f5b040b5440 2024-09-10T22:59:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.868+0000 7f5b1cc00640 1 -- 172.21.15.67:0/684503923 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340920 (secure 0 0 0) 0x7f5b20063590 con 0x7f5b040b5440 2024-09-10T22:59:31.872 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:31.874 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 msgr2=0x7f5b040b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 0x7f5b040b7900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005fd0 tx=0x7f5b1000b390 comp rx=0 tx=0).stop 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 msgr2=0x7f5b201a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800cce0 tx=0x7f5b1800a620 comp rx=0 tx=0).stop 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 shutdown_connections 2024-09-10T22:59:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5b040b5440 0x7f5b040b7900 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20109130 0x7f5b201a0c10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b201087b0 0x7f5b2019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 --2- 172.21.15.67:0/684503923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b20102da0 0x7f5b2019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 >> 172.21.15.67:0/684503923 conn(0x7f5b200fe810 msgr2=0x7f5b20104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 shutdown_connections 2024-09-10T22:59:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:31.873+0000 7f5b27200640 1 -- 172.21.15.67:0/684503923 wait complete. 2024-09-10T22:59:32.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:32 smithi067 ceph-mon[30939]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:32.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:32 smithi145 ceph-mon[34397]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:32.656 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T22:59:30.108393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1445536,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1254,"num_read_kb":861,"num_write":534,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":894,"ondisk_log_size":894,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357332,"kb_used_data":12208,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584492,"statfs":{"total":1151940427776,"available":1151574519808,"internally_reserved":0,"allocated":12500992,"data_stored":10541606,"data_compressed":575187,"data_compressed_allocated":774144,"data_compressed_original":5038080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":788471,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":412,"num_read_kb":337,"num_write":234,"num_write_kb":2096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003288"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184942+0000","last_change":"2024-09-10T22:56:16.894878+0000","last_active":"2024-09-10T22:58:34.184942+0000","last_peered":"2024-09-10T22:58:34.184942+0000","last_clean":"2024-09-10T22:58:34.184942+0000","last_became_active":"2024-09-10T22:56:16.894421+0000","last_became_peered":"2024-09-10T22:56:16.894421+0000","last_unstale":"2024-09-10T22:58:34.184942+0000","last_undegraded":"2024-09-10T22:58:34.184942+0000","last_fullsized":"2024-09-10T22:58:34.184942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:00.251912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462913+0000","last_change":"2024-09-10T22:56:18.915385+0000","last_active":"2024-09-10T22:58:34.462913+0000","last_peered":"2024-09-10T22:58:34.462913+0000","last_clean":"2024-09-10T22:58:34.462913+0000","last_became_active":"2024-09-10T22:56:18.914975+0000","last_became_peered":"2024-09-10T22:56:18.914975+0000","last_unstale":"2024-09-10T22:58:34.462913+0000","last_undegraded":"2024-09-10T22:58:34.462913+0000","last_fullsized":"2024-09-10T22:58:34.462913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:28.978645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188615+0000","last_change":"2024-09-10T22:56:27.963760+0000","last_active":"2024-09-10T22:58:34.188615+0000","last_peered":"2024-09-10T22:58:34.188615+0000","last_clean":"2024-09-10T22:58:34.188615+0000","last_became_active":"2024-09-10T22:56:27.963379+0000","last_became_peered":"2024-09-10T22:56:27.963379+0000","last_unstale":"2024-09-10T22:58:34.188615+0000","last_undegraded":"2024-09-10T22:58:34.188615+0000","last_fullsized":"2024-09-10T22:58:34.188615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:59.721176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463180+0000","last_change":"2024-09-10T22:56:17.604955+0000","last_active":"2024-09-10T22:58:34.463180+0000","last_peered":"2024-09-10T22:58:34.463180+0000","last_clean":"2024-09-10T22:58:34.463180+0000","last_became_active":"2024-09-10T22:56:17.604586+0000","last_became_peered":"2024-09-10T22:56:17.604586+0000","last_unstale":"2024-09-10T22:58:34.463180+0000","last_undegraded":"2024-09-10T22:58:34.463180+0000","last_fullsized":"2024-09-10T22:58:34.463180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:51.268699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666721+0000","last_change":"2024-09-10T22:56:18.920646+0000","last_active":"2024-09-10T22:58:34.666721+0000","last_peered":"2024-09-10T22:58:34.666721+0000","last_clean":"2024-09-10T22:58:34.666721+0000","last_became_active":"2024-09-10T22:56:18.920219+0000","last_became_peered":"2024-09-10T22:56:18.920219+0000","last_unstale":"2024-09-10T22:58:34.666721+0000","last_undegraded":"2024-09-10T22:58:34.666721+0000","last_fullsized":"2024-09-10T22:58:34.666721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:26.596692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187313+0000","last_change":"2024-09-10T22:56:27.963802+0000","last_active":"2024-09-10T22:58:34.187313+0000","last_peered":"2024-09-10T22:58:34.187313+0000","last_clean":"2024-09-10T22:58:34.187313+0000","last_became_active":"2024-09-10T22:56:27.963413+0000","last_became_peered":"2024-09-10T22:56:27.963413+0000","last_unstale":"2024-09-10T22:58:34.187313+0000","last_undegraded":"2024-09-10T22:58:34.187313+0000","last_fullsized":"2024-09-10T22:58:34.187313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:46.594317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465001+0000","last_change":"2024-09-10T22:56:16.898708+0000","last_active":"2024-09-10T22:58:34.465001+0000","last_peered":"2024-09-10T22:58:34.465001+0000","last_clean":"2024-09-10T22:58:34.465001+0000","last_became_active":"2024-09-10T22:56:16.897681+0000","last_became_peered":"2024-09-10T22:56:16.897681+0000","last_unstale":"2024-09-10T22:58:34.465001+0000","last_undegraded":"2024-09-10T22:58:34.465001+0000","last_fullsized":"2024-09-10T22:58:34.465001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.883254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587823+0000","last_change":"2024-09-10T22:56:18.920849+0000","last_active":"2024-09-10T22:58:34.587823+0000","last_peered":"2024-09-10T22:58:34.587823+0000","last_clean":"2024-09-10T22:58:34.587823+0000","last_became_active":"2024-09-10T22:56:18.920472+0000","last_became_peered":"2024-09-10T22:56:18.920472+0000","last_unstale":"2024-09-10T22:58:34.587823+0000","last_undegraded":"2024-09-10T22:58:34.587823+0000","last_fullsized":"2024-09-10T22:58:34.587823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:20.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188011+0000","last_change":"2024-09-10T22:56:27.968028+0000","last_active":"2024-09-10T22:58:34.188011+0000","last_peered":"2024-09-10T22:58:34.188011+0000","last_clean":"2024-09-10T22:58:34.188011+0000","last_became_active":"2024-09-10T22:56:27.967543+0000","last_became_peered":"2024-09-10T22:56:27.967543+0000","last_unstale":"2024-09-10T22:58:34.188011+0000","last_undegraded":"2024-09-10T22:58:34.188011+0000","last_fullsized":"2024-09-10T22:58:34.188011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.248146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186395+0000","last_change":"2024-09-10T22:57:33.499653+0000","last_active":"2024-09-10T22:58:34.186395+0000","last_peered":"2024-09-10T22:58:34.186395+0000","last_clean":"2024-09-10T22:58:34.186395+0000","last_became_active":"2024-09-10T22:57:33.492484+0000","last_became_peered":"2024-09-10T22:57:33.492484+0000","last_unstale":"2024-09-10T22:58:34.186395+0000","last_undegraded":"2024-09-10T22:58:34.186395+0000","last_fullsized":"2024-09-10T22:58:34.186395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:05.379683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602782+0000","last_change":"2024-09-10T22:57:33.637481+0000","last_active":"2024-09-10T22:58:34.602782+0000","last_peered":"2024-09-10T22:58:34.602782+0000","last_clean":"2024-09-10T22:58:34.602782+0000","last_became_active":"2024-09-10T22:57:33.637143+0000","last_became_peered":"2024-09-10T22:57:33.637143+0000","last_unstale":"2024-09-10T22:58:34.602782+0000","last_undegraded":"2024-09-10T22:58:34.602782+0000","last_fullsized":"2024-09-10T22:58:34.602782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:49.501830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187578+0000","last_change":"2024-09-10T22:56:27.957761+0000","last_active":"2024-09-10T22:58:34.187578+0000","last_peered":"2024-09-10T22:58:34.187578+0000","last_clean":"2024-09-10T22:58:34.187578+0000","last_became_active":"2024-09-10T22:56:27.957088+0000","last_became_peered":"2024-09-10T22:56:27.957088+0000","last_unstale":"2024-09-10T22:58:34.187578+0000","last_undegraded":"2024-09-10T22:58:34.187578+0000","last_fullsized":"2024-09-10T22:58:34.187578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:03.942109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185664+0000","last_change":"2024-09-10T22:56:18.927781+0000","last_active":"2024-09-10T22:58:34.185664+0000","last_peered":"2024-09-10T22:58:34.185664+0000","last_clean":"2024-09-10T22:58:34.185664+0000","last_became_active":"2024-09-10T22:56:18.927466+0000","last_became_peered":"2024-09-10T22:56:18.927466+0000","last_unstale":"2024-09-10T22:58:34.185664+0000","last_undegraded":"2024-09-10T22:58:34.185664+0000","last_fullsized":"2024-09-10T22:58:34.185664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:49.198677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464494+0000","last_change":"2024-09-10T22:56:16.896733+0000","last_active":"2024-09-10T22:58:34.464494+0000","last_peered":"2024-09-10T22:58:34.464494+0000","last_clean":"2024-09-10T22:58:34.464494+0000","last_became_active":"2024-09-10T22:56:16.896272+0000","last_became_peered":"2024-09-10T22:56:16.896272+0000","last_unstale":"2024-09-10T22:58:34.464494+0000","last_undegraded":"2024-09-10T22:58:34.464494+0000","last_fullsized":"2024-09-10T22:58:34.464494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:47.380845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187941+0000","last_change":"2024-09-10T22:56:27.959672+0000","last_active":"2024-09-10T22:58:34.187941+0000","last_peered":"2024-09-10T22:58:34.187941+0000","last_clean":"2024-09-10T22:58:34.187941+0000","last_became_active":"2024-09-10T22:56:27.957424+0000","last_became_peered":"2024-09-10T22:56:27.957424+0000","last_unstale":"2024-09-10T22:58:34.187941+0000","last_undegraded":"2024-09-10T22:58:34.187941+0000","last_fullsized":"2024-09-10T22:58:34.187941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:31.204239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462839+0000","last_change":"2024-09-10T22:56:18.917490+0000","last_active":"2024-09-10T22:58:34.462839+0000","last_peered":"2024-09-10T22:58:34.462839+0000","last_clean":"2024-09-10T22:58:34.462839+0000","last_became_active":"2024-09-10T22:56:18.916498+0000","last_became_peered":"2024-09-10T22:56:18.916498+0000","last_unstale":"2024-09-10T22:58:34.462839+0000","last_undegraded":"2024-09-10T22:58:34.462839+0000","last_fullsized":"2024-09-10T22:58:34.462839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:53.737755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188745+0000","last_change":"2024-09-10T22:56:16.893935+0000","last_active":"2024-09-10T22:58:34.188745+0000","last_peered":"2024-09-10T22:58:34.188745+0000","last_clean":"2024-09-10T22:58:34.188745+0000","last_became_active":"2024-09-10T22:56:16.893537+0000","last_became_peered":"2024-09-10T22:56:16.893537+0000","last_unstale":"2024-09-10T22:58:34.188745+0000","last_undegraded":"2024-09-10T22:58:34.188745+0000","last_fullsized":"2024-09-10T22:58:34.188745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:23.150759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187786+0000","last_change":"2024-09-10T22:56:27.964839+0000","last_active":"2024-09-10T22:58:34.187786+0000","last_peered":"2024-09-10T22:58:34.187786+0000","last_clean":"2024-09-10T22:58:34.187786+0000","last_became_active":"2024-09-10T22:56:27.964420+0000","last_became_peered":"2024-09-10T22:56:27.964420+0000","last_unstale":"2024-09-10T22:58:34.187786+0000","last_undegraded":"2024-09-10T22:58:34.187786+0000","last_fullsized":"2024-09-10T22:58:34.187786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186083+0000","last_change":"2024-09-10T22:56:18.918189+0000","last_active":"2024-09-10T22:58:34.186083+0000","last_peered":"2024-09-10T22:58:34.186083+0000","last_clean":"2024-09-10T22:58:34.186083+0000","last_became_active":"2024-09-10T22:56:18.917791+0000","last_became_peered":"2024-09-10T22:56:18.917791+0000","last_unstale":"2024-09-10T22:58:34.186083+0000","last_undegraded":"2024-09-10T22:58:34.186083+0000","last_fullsized":"2024-09-10T22:58:34.186083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:11.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462767+0000","last_change":"2024-09-10T22:56:16.889746+0000","last_active":"2024-09-10T22:58:34.462767+0000","last_peered":"2024-09-10T22:58:34.462767+0000","last_clean":"2024-09-10T22:58:34.462767+0000","last_became_active":"2024-09-10T22:56:16.888366+0000","last_became_peered":"2024-09-10T22:56:16.888366+0000","last_unstale":"2024-09-10T22:58:34.462767+0000","last_undegraded":"2024-09-10T22:58:34.462767+0000","last_fullsized":"2024-09-10T22:58:34.462767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:09.479447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184820+0000","last_change":"2024-09-10T22:56:27.958135+0000","last_active":"2024-09-10T22:58:34.184820+0000","last_peered":"2024-09-10T22:58:34.184820+0000","last_clean":"2024-09-10T22:58:34.184820+0000","last_became_active":"2024-09-10T22:56:27.954714+0000","last_became_peered":"2024-09-10T22:56:27.954714+0000","last_unstale":"2024-09-10T22:58:34.184820+0000","last_undegraded":"2024-09-10T22:58:34.184820+0000","last_fullsized":"2024-09-10T22:58:34.184820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:56.731655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464201+0000","last_change":"2024-09-10T22:56:18.916361+0000","last_active":"2024-09-10T22:58:34.464201+0000","last_peered":"2024-09-10T22:58:34.464201+0000","last_clean":"2024-09-10T22:58:34.464201+0000","last_became_active":"2024-09-10T22:56:18.915957+0000","last_became_peered":"2024-09-10T22:56:18.915957+0000","last_unstale":"2024-09-10T22:58:34.464201+0000","last_undegraded":"2024-09-10T22:58:34.464201+0000","last_fullsized":"2024-09-10T22:58:34.464201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:20.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587613+0000","last_change":"2024-09-10T22:56:17.606445+0000","last_active":"2024-09-10T22:58:34.587613+0000","last_peered":"2024-09-10T22:58:34.587613+0000","last_clean":"2024-09-10T22:58:34.587613+0000","last_became_active":"2024-09-10T22:56:17.606018+0000","last_became_peered":"2024-09-10T22:56:17.606018+0000","last_unstale":"2024-09-10T22:58:34.587613+0000","last_undegraded":"2024-09-10T22:58:34.587613+0000","last_fullsized":"2024-09-10T22:58:34.587613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:20.493370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466740+0000","last_change":"2024-09-10T22:58:34.466034+0000","last_active":"2024-09-10T22:58:34.466740+0000","last_peered":"2024-09-10T22:58:34.466740+0000","last_clean":"2024-09-10T22:58:34.466740+0000","last_became_active":"2024-09-10T22:58:34.465493+0000","last_became_peered":"2024-09-10T22:58:34.465493+0000","last_unstale":"2024-09-10T22:58:34.466740+0000","last_undegraded":"2024-09-10T22:58:34.466740+0000","last_fullsized":"2024-09-10T22:58:34.466740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:43.360753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666497+0000","last_change":"2024-09-10T22:56:18.926231+0000","last_active":"2024-09-10T22:58:34.666497+0000","last_peered":"2024-09-10T22:58:34.666497+0000","last_clean":"2024-09-10T22:58:34.666497+0000","last_became_active":"2024-09-10T22:56:18.923621+0000","last_became_peered":"2024-09-10T22:56:18.923621+0000","last_unstale":"2024-09-10T22:58:34.666497+0000","last_undegraded":"2024-09-10T22:58:34.666497+0000","last_fullsized":"2024-09-10T22:58:34.666497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:23.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188622+0000","last_change":"2024-09-10T22:56:16.891129+0000","last_active":"2024-09-10T22:58:34.188622+0000","last_peered":"2024-09-10T22:58:34.188622+0000","last_clean":"2024-09-10T22:58:34.188622+0000","last_became_active":"2024-09-10T22:56:16.890473+0000","last_became_peered":"2024-09-10T22:56:16.890473+0000","last_unstale":"2024-09-10T22:58:34.188622+0000","last_undegraded":"2024-09-10T22:58:34.188622+0000","last_fullsized":"2024-09-10T22:58:34.188622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:02.422979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188981+0000","last_change":"2024-09-10T22:56:27.960059+0000","last_active":"2024-09-10T22:58:34.188981+0000","last_peered":"2024-09-10T22:58:34.188981+0000","last_clean":"2024-09-10T22:58:34.188981+0000","last_became_active":"2024-09-10T22:56:27.959191+0000","last_became_peered":"2024-09-10T22:56:27.959191+0000","last_unstale":"2024-09-10T22:58:34.188981+0000","last_undegraded":"2024-09-10T22:58:34.188981+0000","last_fullsized":"2024-09-10T22:58:34.188981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:44.884133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.463075+0000","last_change":"2024-09-10T22:56:18.923529+0000","last_active":"2024-09-10T22:58:34.463075+0000","last_peered":"2024-09-10T22:58:34.463075+0000","last_clean":"2024-09-10T22:58:34.463075+0000","last_became_active":"2024-09-10T22:56:18.923119+0000","last_became_peered":"2024-09-10T22:56:18.923119+0000","last_unstale":"2024-09-10T22:58:34.463075+0000","last_undegraded":"2024-09-10T22:58:34.463075+0000","last_fullsized":"2024-09-10T22:58:34.463075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:12.441237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'97","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:26.296239+0000","last_change":"2024-09-10T22:56:16.888699+0000","last_active":"2024-09-10T22:59:26.296239+0000","last_peered":"2024-09-10T22:59:26.296239+0000","last_clean":"2024-09-10T22:59:26.296239+0000","last_became_active":"2024-09-10T22:56:16.888311+0000","last_became_peered":"2024-09-10T22:56:16.888311+0000","last_unstale":"2024-09-10T22:59:26.296239+0000","last_undegraded":"2024-09-10T22:59:26.296239+0000","last_fullsized":"2024-09-10T22:59:26.296239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:16.597171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462341+0000","last_change":"2024-09-10T22:56:27.951578+0000","last_active":"2024-09-10T22:58:34.462341+0000","last_peered":"2024-09-10T22:58:34.462341+0000","last_clean":"2024-09-10T22:58:34.462341+0000","last_became_active":"2024-09-10T22:56:27.950690+0000","last_became_peered":"2024-09-10T22:56:27.950690+0000","last_unstale":"2024-09-10T22:58:34.462341+0000","last_undegraded":"2024-09-10T22:58:34.462341+0000","last_fullsized":"2024-09-10T22:58:34.462341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:52.135578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186602+0000","last_change":"2024-09-10T22:56:18.922679+0000","last_active":"2024-09-10T22:58:34.186602+0000","last_peered":"2024-09-10T22:58:34.186602+0000","last_clean":"2024-09-10T22:58:34.186602+0000","last_became_active":"2024-09-10T22:56:18.921454+0000","last_became_peered":"2024-09-10T22:56:18.921454+0000","last_unstale":"2024-09-10T22:58:34.186602+0000","last_undegraded":"2024-09-10T22:58:34.186602+0000","last_fullsized":"2024-09-10T22:58:34.186602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:35.610764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464970+0000","last_change":"2024-09-10T22:56:17.606379+0000","last_active":"2024-09-10T22:58:34.464970+0000","last_peered":"2024-09-10T22:58:34.464970+0000","last_clean":"2024-09-10T22:58:34.464970+0000","last_became_active":"2024-09-10T22:56:17.606015+0000","last_became_peered":"2024-09-10T22:56:17.606015+0000","last_unstale":"2024-09-10T22:58:34.464970+0000","last_undegraded":"2024-09-10T22:58:34.464970+0000","last_fullsized":"2024-09-10T22:58:34.464970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:09.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185884+0000","last_change":"2024-09-10T22:56:27.953730+0000","last_active":"2024-09-10T22:58:34.185884+0000","last_peered":"2024-09-10T22:58:34.185884+0000","last_clean":"2024-09-10T22:58:34.185884+0000","last_became_active":"2024-09-10T22:56:27.952725+0000","last_became_peered":"2024-09-10T22:56:27.952725+0000","last_unstale":"2024-09-10T22:58:34.185884+0000","last_undegraded":"2024-09-10T22:58:34.185884+0000","last_fullsized":"2024-09-10T22:58:34.185884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:42.791249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189026+0000","last_change":"2024-09-10T22:56:18.913714+0000","last_active":"2024-09-10T22:58:34.189026+0000","last_peered":"2024-09-10T22:58:34.189026+0000","last_clean":"2024-09-10T22:58:34.189026+0000","last_became_active":"2024-09-10T22:56:18.913107+0000","last_became_peered":"2024-09-10T22:56:18.913107+0000","last_unstale":"2024-09-10T22:58:34.189026+0000","last_undegraded":"2024-09-10T22:58:34.189026+0000","last_fullsized":"2024-09-10T22:58:34.189026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:56.181287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184648+0000","last_change":"2024-09-10T22:56:17.606877+0000","last_active":"2024-09-10T22:58:34.184648+0000","last_peered":"2024-09-10T22:58:34.184648+0000","last_clean":"2024-09-10T22:58:34.184648+0000","last_became_active":"2024-09-10T22:56:17.606098+0000","last_became_peered":"2024-09-10T22:56:17.606098+0000","last_unstale":"2024-09-10T22:58:34.184648+0000","last_undegraded":"2024-09-10T22:58:34.184648+0000","last_fullsized":"2024-09-10T22:58:34.184648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:31.033832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464415+0000","last_change":"2024-09-10T22:56:27.964297+0000","last_active":"2024-09-10T22:58:34.464415+0000","last_peered":"2024-09-10T22:58:34.464415+0000","last_clean":"2024-09-10T22:58:34.464415+0000","last_became_active":"2024-09-10T22:56:27.963484+0000","last_became_peered":"2024-09-10T22:56:27.963484+0000","last_unstale":"2024-09-10T22:58:34.464415+0000","last_undegraded":"2024-09-10T22:58:34.464415+0000","last_fullsized":"2024-09-10T22:58:34.464415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:10.603044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188871+0000","last_change":"2024-09-10T22:56:18.923784+0000","last_active":"2024-09-10T22:58:34.188871+0000","last_peered":"2024-09-10T22:58:34.188871+0000","last_clean":"2024-09-10T22:58:34.188871+0000","last_became_active":"2024-09-10T22:56:18.923480+0000","last_became_peered":"2024-09-10T22:56:18.923480+0000","last_unstale":"2024-09-10T22:58:34.188871+0000","last_undegraded":"2024-09-10T22:58:34.188871+0000","last_fullsized":"2024-09-10T22:58:34.188871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:37.678045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185784+0000","last_change":"2024-09-10T22:56:16.893522+0000","last_active":"2024-09-10T22:58:34.185784+0000","last_peered":"2024-09-10T22:58:34.185784+0000","last_clean":"2024-09-10T22:58:34.185784+0000","last_became_active":"2024-09-10T22:56:16.893060+0000","last_became_peered":"2024-09-10T22:56:16.893060+0000","last_unstale":"2024-09-10T22:58:34.185784+0000","last_undegraded":"2024-09-10T22:58:34.185784+0000","last_fullsized":"2024-09-10T22:58:34.185784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:38.789963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186540+0000","last_change":"2024-09-10T22:56:27.960726+0000","last_active":"2024-09-10T22:58:34.186540+0000","last_peered":"2024-09-10T22:58:34.186540+0000","last_clean":"2024-09-10T22:58:34.186540+0000","last_became_active":"2024-09-10T22:56:27.960203+0000","last_became_peered":"2024-09-10T22:56:27.960203+0000","last_unstale":"2024-09-10T22:58:34.186540+0000","last_undegraded":"2024-09-10T22:58:34.186540+0000","last_fullsized":"2024-09-10T22:58:34.186540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:55.102324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602469+0000","last_change":"2024-09-10T22:56:18.922265+0000","last_active":"2024-09-10T22:58:34.602469+0000","last_peered":"2024-09-10T22:58:34.602469+0000","last_clean":"2024-09-10T22:58:34.602469+0000","last_became_active":"2024-09-10T22:56:18.921895+0000","last_became_peered":"2024-09-10T22:56:18.921895+0000","last_unstale":"2024-09-10T22:58:34.602469+0000","last_undegraded":"2024-09-10T22:58:34.602469+0000","last_fullsized":"2024-09-10T22:58:34.602469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:25.861320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:28.705336+0000","last_change":"2024-09-10T22:56:17.605055+0000","last_active":"2024-09-10T22:59:28.705336+0000","last_peered":"2024-09-10T22:59:28.705336+0000","last_clean":"2024-09-10T22:59:28.705336+0000","last_became_active":"2024-09-10T22:56:17.604693+0000","last_became_peered":"2024-09-10T22:56:17.604693+0000","last_unstale":"2024-09-10T22:59:28.705336+0000","last_undegraded":"2024-09-10T22:59:28.705336+0000","last_fullsized":"2024-09-10T22:59:28.705336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:51.778409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186783+0000","last_change":"2024-09-10T22:56:27.953268+0000","last_active":"2024-09-10T22:58:34.186783+0000","last_peered":"2024-09-10T22:58:34.186783+0000","last_clean":"2024-09-10T22:58:34.186783+0000","last_became_active":"2024-09-10T22:56:27.952801+0000","last_became_peered":"2024-09-10T22:56:27.952801+0000","last_unstale":"2024-09-10T22:58:34.186783+0000","last_undegraded":"2024-09-10T22:58:34.186783+0000","last_fullsized":"2024-09-10T22:58:34.186783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:04.260497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.190954+0000","last_change":"2024-09-10T22:58:34.190886+0000","last_active":"2024-09-10T22:58:34.190954+0000","last_peered":"2024-09-10T22:58:34.190954+0000","last_clean":"2024-09-10T22:58:34.190954+0000","last_became_active":"2024-09-10T22:58:34.190534+0000","last_became_peered":"2024-09-10T22:58:34.190534+0000","last_unstale":"2024-09-10T22:58:34.190954+0000","last_undegraded":"2024-09-10T22:58:34.190954+0000","last_fullsized":"2024-09-10T22:58:34.190954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:07.888931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185571+0000","last_change":"2024-09-10T22:56:16.898855+0000","last_active":"2024-09-10T22:58:34.185571+0000","last_peered":"2024-09-10T22:58:34.185571+0000","last_clean":"2024-09-10T22:58:34.185571+0000","last_became_active":"2024-09-10T22:56:16.898330+0000","last_became_peered":"2024-09-10T22:56:16.898330+0000","last_unstale":"2024-09-10T22:58:34.185571+0000","last_undegraded":"2024-09-10T22:58:34.185571+0000","last_fullsized":"2024-09-10T22:58:34.185571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:29.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.603153+0000","last_change":"2024-09-10T22:56:27.963379+0000","last_active":"2024-09-10T22:58:34.603153+0000","last_peered":"2024-09-10T22:58:34.603153+0000","last_clean":"2024-09-10T22:58:34.603153+0000","last_became_active":"2024-09-10T22:56:27.962949+0000","last_became_peered":"2024-09-10T22:56:27.962949+0000","last_unstale":"2024-09-10T22:58:34.603153+0000","last_undegraded":"2024-09-10T22:58:34.603153+0000","last_fullsized":"2024-09-10T22:58:34.603153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:52.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184347+0000","last_change":"2024-09-10T22:57:33.488910+0000","last_active":"2024-09-10T22:58:34.184347+0000","last_peered":"2024-09-10T22:58:34.184347+0000","last_clean":"2024-09-10T22:58:34.184347+0000","last_became_active":"2024-09-10T22:57:33.488668+0000","last_became_peered":"2024-09-10T22:57:33.488668+0000","last_unstale":"2024-09-10T22:58:34.184347+0000","last_undegraded":"2024-09-10T22:58:34.184347+0000","last_fullsized":"2024-09-10T22:58:34.184347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:20.013788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188099+0000","last_change":"2024-09-10T22:56:16.891650+0000","last_active":"2024-09-10T22:58:34.188099+0000","last_peered":"2024-09-10T22:58:34.188099+0000","last_clean":"2024-09-10T22:58:34.188099+0000","last_became_active":"2024-09-10T22:56:16.891162+0000","last_became_peered":"2024-09-10T22:56:16.891162+0000","last_unstale":"2024-09-10T22:58:34.188099+0000","last_undegraded":"2024-09-10T22:58:34.188099+0000","last_fullsized":"2024-09-10T22:58:34.188099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:51.044793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185870+0000","last_change":"2024-09-10T22:54:46.884225+0000","last_active":"2024-09-10T22:58:34.185870+0000","last_peered":"2024-09-10T22:58:34.185870+0000","last_clean":"2024-09-10T22:58:34.185870+0000","last_became_active":"2024-09-10T22:54:46.878937+0000","last_became_peered":"2024-09-10T22:54:46.878937+0000","last_unstale":"2024-09-10T22:58:34.185870+0000","last_undegraded":"2024-09-10T22:58:34.185870+0000","last_fullsized":"2024-09-10T22:58:34.185870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:44.451159+0000","last_clean_scrub_stamp":"2024-09-10T22:50:44.451159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:18.007567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186444+0000","last_change":"2024-09-10T22:56:27.961865+0000","last_active":"2024-09-10T22:58:34.186444+0000","last_peered":"2024-09-10T22:58:34.186444+0000","last_clean":"2024-09-10T22:58:34.186444+0000","last_became_active":"2024-09-10T22:56:27.961419+0000","last_became_peered":"2024-09-10T22:56:27.961419+0000","last_unstale":"2024-09-10T22:58:34.186444+0000","last_undegraded":"2024-09-10T22:58:34.186444+0000","last_fullsized":"2024-09-10T22:58:34.186444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:43.088570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186412+0000","last_change":"2024-09-10T22:56:18.920994+0000","last_active":"2024-09-10T22:58:34.186412+0000","last_peered":"2024-09-10T22:58:34.186412+0000","last_clean":"2024-09-10T22:58:34.186412+0000","last_became_active":"2024-09-10T22:56:18.920620+0000","last_became_peered":"2024-09-10T22:56:18.920620+0000","last_unstale":"2024-09-10T22:58:34.186412+0000","last_undegraded":"2024-09-10T22:58:34.186412+0000","last_fullsized":"2024-09-10T22:58:34.186412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:40.503444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189203+0000","last_change":"2024-09-10T22:56:17.607536+0000","last_active":"2024-09-10T22:58:34.189203+0000","last_peered":"2024-09-10T22:58:34.189203+0000","last_clean":"2024-09-10T22:58:34.189203+0000","last_became_active":"2024-09-10T22:56:17.607190+0000","last_became_peered":"2024-09-10T22:56:17.607190+0000","last_unstale":"2024-09-10T22:58:34.189203+0000","last_undegraded":"2024-09-10T22:58:34.189203+0000","last_fullsized":"2024-09-10T22:58:34.189203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:12.683248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587205+0000","last_change":"2024-09-10T22:56:27.963487+0000","last_active":"2024-09-10T22:58:34.587205+0000","last_peered":"2024-09-10T22:58:34.587205+0000","last_clean":"2024-09-10T22:58:34.587205+0000","last_became_active":"2024-09-10T22:56:27.963047+0000","last_became_peered":"2024-09-10T22:56:27.963047+0000","last_unstale":"2024-09-10T22:58:34.587205+0000","last_undegraded":"2024-09-10T22:58:34.587205+0000","last_fullsized":"2024-09-10T22:58:34.587205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:50.654952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185988+0000","last_change":"2024-09-10T22:57:33.495738+0000","last_active":"2024-09-10T22:58:34.185988+0000","last_peered":"2024-09-10T22:58:34.185988+0000","last_clean":"2024-09-10T22:58:34.185988+0000","last_became_active":"2024-09-10T22:57:33.495236+0000","last_became_peered":"2024-09-10T22:57:33.495236+0000","last_unstale":"2024-09-10T22:58:34.185988+0000","last_undegraded":"2024-09-10T22:58:34.185988+0000","last_fullsized":"2024-09-10T22:58:34.185988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:22.324087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'60","reported_seq":130,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:25.883462+0000","last_change":"2024-09-10T22:56:16.899196+0000","last_active":"2024-09-10T22:59:25.883462+0000","last_peered":"2024-09-10T22:59:25.883462+0000","last_clean":"2024-09-10T22:59:25.883462+0000","last_became_active":"2024-09-10T22:56:16.897796+0000","last_became_peered":"2024-09-10T22:56:16.897796+0000","last_unstale":"2024-09-10T22:59:25.883462+0000","last_undegraded":"2024-09-10T22:59:25.883462+0000","last_fullsized":"2024-09-10T22:59:25.883462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:52.520406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666303+0000","last_change":"2024-09-10T22:56:27.961569+0000","last_active":"2024-09-10T22:58:34.666303+0000","last_peered":"2024-09-10T22:58:34.666303+0000","last_clean":"2024-09-10T22:58:34.666303+0000","last_became_active":"2024-09-10T22:56:27.961136+0000","last_became_peered":"2024-09-10T22:56:27.961136+0000","last_unstale":"2024-09-10T22:58:34.666303+0000","last_undegraded":"2024-09-10T22:58:34.666303+0000","last_fullsized":"2024-09-10T22:58:34.666303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:42.348661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186147+0000","last_change":"2024-09-10T22:56:18.922247+0000","last_active":"2024-09-10T22:58:34.186147+0000","last_peered":"2024-09-10T22:58:34.186147+0000","last_clean":"2024-09-10T22:58:34.186147+0000","last_became_active":"2024-09-10T22:56:18.921387+0000","last_became_peered":"2024-09-10T22:56:18.921387+0000","last_unstale":"2024-09-10T22:58:34.186147+0000","last_undegraded":"2024-09-10T22:58:34.186147+0000","last_fullsized":"2024-09-10T22:58:34.186147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:53.198483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187257+0000","last_change":"2024-09-10T22:56:16.897308+0000","last_active":"2024-09-10T22:58:34.187257+0000","last_peered":"2024-09-10T22:58:34.187257+0000","last_clean":"2024-09-10T22:58:34.187257+0000","last_became_active":"2024-09-10T22:56:16.896876+0000","last_became_peered":"2024-09-10T22:56:16.896876+0000","last_unstale":"2024-09-10T22:58:34.187257+0000","last_undegraded":"2024-09-10T22:58:34.187257+0000","last_fullsized":"2024-09-10T22:58:34.187257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:42.421198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.601976+0000","last_change":"2024-09-10T22:56:27.958458+0000","last_active":"2024-09-10T22:58:34.601976+0000","last_peered":"2024-09-10T22:58:34.601976+0000","last_clean":"2024-09-10T22:58:34.601976+0000","last_became_active":"2024-09-10T22:56:27.957880+0000","last_became_peered":"2024-09-10T22:56:27.957880+0000","last_unstale":"2024-09-10T22:58:34.601976+0000","last_undegraded":"2024-09-10T22:58:34.601976+0000","last_fullsized":"2024-09-10T22:58:34.601976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:08.234968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186276+0000","last_change":"2024-09-10T22:56:18.924731+0000","last_active":"2024-09-10T22:58:34.186276+0000","last_peered":"2024-09-10T22:58:34.186276+0000","last_clean":"2024-09-10T22:58:34.186276+0000","last_became_active":"2024-09-10T22:56:18.924402+0000","last_became_peered":"2024-09-10T22:56:18.924402+0000","last_unstale":"2024-09-10T22:58:34.186276+0000","last_undegraded":"2024-09-10T22:58:34.186276+0000","last_fullsized":"2024-09-10T22:58:34.186276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:35.005305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.465088+0000","last_change":"2024-09-10T22:56:16.890767+0000","last_active":"2024-09-10T22:58:34.465088+0000","last_peered":"2024-09-10T22:58:34.465088+0000","last_clean":"2024-09-10T22:58:34.465088+0000","last_became_active":"2024-09-10T22:56:16.889871+0000","last_became_peered":"2024-09-10T22:56:16.889871+0000","last_unstale":"2024-09-10T22:58:34.465088+0000","last_undegraded":"2024-09-10T22:58:34.465088+0000","last_fullsized":"2024-09-10T22:58:34.465088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:42.302703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602472+0000","last_change":"2024-09-10T22:56:27.960322+0000","last_active":"2024-09-10T22:58:34.602472+0000","last_peered":"2024-09-10T22:58:34.602472+0000","last_clean":"2024-09-10T22:58:34.602472+0000","last_became_active":"2024-09-10T22:56:27.959794+0000","last_became_peered":"2024-09-10T22:56:27.959794+0000","last_unstale":"2024-09-10T22:58:34.602472+0000","last_undegraded":"2024-09-10T22:58:34.602472+0000","last_fullsized":"2024-09-10T22:58:34.602472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:28.327737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602600+0000","last_change":"2024-09-10T22:57:33.490252+0000","last_active":"2024-09-10T22:58:34.602600+0000","last_peered":"2024-09-10T22:58:34.602600+0000","last_clean":"2024-09-10T22:58:34.602600+0000","last_became_active":"2024-09-10T22:57:33.489871+0000","last_became_peered":"2024-09-10T22:57:33.489871+0000","last_unstale":"2024-09-10T22:58:34.602600+0000","last_undegraded":"2024-09-10T22:58:34.602600+0000","last_fullsized":"2024-09-10T22:58:34.602600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:23.075920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185766+0000","last_change":"2024-09-10T22:57:33.968383+0000","last_active":"2024-09-10T22:58:34.185766+0000","last_peered":"2024-09-10T22:58:34.185766+0000","last_clean":"2024-09-10T22:58:34.185766+0000","last_became_active":"2024-09-10T22:57:33.935361+0000","last_became_peered":"2024-09-10T22:57:33.935361+0000","last_unstale":"2024-09-10T22:58:34.185766+0000","last_undegraded":"2024-09-10T22:58:34.185766+0000","last_fullsized":"2024-09-10T22:58:34.185766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:05.211164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.466980+0000","last_change":"2024-09-10T22:58:34.466291+0000","last_active":"2024-09-10T22:58:34.466980+0000","last_peered":"2024-09-10T22:58:34.466980+0000","last_clean":"2024-09-10T22:58:34.466980+0000","last_became_active":"2024-09-10T22:58:34.465710+0000","last_became_peered":"2024-09-10T22:58:34.465710+0000","last_unstale":"2024-09-10T22:58:34.466980+0000","last_undegraded":"2024-09-10T22:58:34.466980+0000","last_fullsized":"2024-09-10T22:58:34.466980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:32.445364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185427+0000","last_change":"2024-09-10T22:57:33.491833+0000","last_active":"2024-09-10T22:58:34.185427+0000","last_peered":"2024-09-10T22:58:34.185427+0000","last_clean":"2024-09-10T22:58:34.185427+0000","last_became_active":"2024-09-10T22:57:33.490951+0000","last_became_peered":"2024-09-10T22:57:33.490951+0000","last_unstale":"2024-09-10T22:58:34.185427+0000","last_undegraded":"2024-09-10T22:58:34.185427+0000","last_fullsized":"2024-09-10T22:58:34.185427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:51.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:27.704036+0000","last_change":"2024-09-10T22:56:16.893903+0000","last_active":"2024-09-10T22:59:27.704036+0000","last_peered":"2024-09-10T22:59:27.704036+0000","last_clean":"2024-09-10T22:59:27.704036+0000","last_became_active":"2024-09-10T22:56:16.893532+0000","last_became_peered":"2024-09-10T22:56:16.893532+0000","last_unstale":"2024-09-10T22:59:27.704036+0000","last_undegraded":"2024-09-10T22:59:27.704036+0000","last_fullsized":"2024-09-10T22:59:27.704036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:53.308538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":141,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184894+0000","last_change":"2024-09-10T22:56:27.958599+0000","last_active":"2024-09-10T22:58:34.184894+0000","last_peered":"2024-09-10T22:58:34.184894+0000","last_clean":"2024-09-10T22:58:34.184894+0000","last_became_active":"2024-09-10T22:56:27.957353+0000","last_became_peered":"2024-09-10T22:56:27.957353+0000","last_unstale":"2024-09-10T22:58:34.184894+0000","last_undegraded":"2024-09-10T22:58:34.184894+0000","last_fullsized":"2024-09-10T22:58:34.184894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:13.093290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185942+0000","last_change":"2024-09-10T22:56:18.925257+0000","last_active":"2024-09-10T22:58:34.185942+0000","last_peered":"2024-09-10T22:58:34.185942+0000","last_clean":"2024-09-10T22:58:34.185942+0000","last_became_active":"2024-09-10T22:56:18.924867+0000","last_became_peered":"2024-09-10T22:56:18.924867+0000","last_unstale":"2024-09-10T22:58:34.185942+0000","last_undegraded":"2024-09-10T22:58:34.185942+0000","last_fullsized":"2024-09-10T22:58:34.185942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:00.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602689+0000","last_change":"2024-09-10T22:56:16.896023+0000","last_active":"2024-09-10T22:58:34.602689+0000","last_peered":"2024-09-10T22:58:34.602689+0000","last_clean":"2024-09-10T22:58:34.602689+0000","last_became_active":"2024-09-10T22:56:16.895562+0000","last_became_peered":"2024-09-10T22:56:16.895562+0000","last_unstale":"2024-09-10T22:58:34.602689+0000","last_undegraded":"2024-09-10T22:58:34.602689+0000","last_fullsized":"2024-09-10T22:58:34.602689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:25.520998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185960+0000","last_change":"2024-09-10T22:56:27.956904+0000","last_active":"2024-09-10T22:58:34.185960+0000","last_peered":"2024-09-10T22:58:34.185960+0000","last_clean":"2024-09-10T22:58:34.185960+0000","last_became_active":"2024-09-10T22:56:27.955738+0000","last_became_peered":"2024-09-10T22:56:27.955738+0000","last_unstale":"2024-09-10T22:58:34.185960+0000","last_undegraded":"2024-09-10T22:58:34.185960+0000","last_fullsized":"2024-09-10T22:58:34.185960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.864356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186578+0000","last_change":"2024-09-10T22:56:18.920128+0000","last_active":"2024-09-10T22:58:34.186578+0000","last_peered":"2024-09-10T22:58:34.186578+0000","last_clean":"2024-09-10T22:58:34.186578+0000","last_became_active":"2024-09-10T22:56:18.919815+0000","last_became_peered":"2024-09-10T22:56:18.919815+0000","last_unstale":"2024-09-10T22:58:34.186578+0000","last_undegraded":"2024-09-10T22:58:34.186578+0000","last_fullsized":"2024-09-10T22:58:34.186578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:57.074964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188921+0000","last_change":"2024-09-10T22:56:16.894046+0000","last_active":"2024-09-10T22:58:34.188921+0000","last_peered":"2024-09-10T22:58:34.188921+0000","last_clean":"2024-09-10T22:58:34.188921+0000","last_became_active":"2024-09-10T22:56:16.893699+0000","last_became_peered":"2024-09-10T22:56:16.893699+0000","last_unstale":"2024-09-10T22:58:34.188921+0000","last_undegraded":"2024-09-10T22:58:34.188921+0000","last_fullsized":"2024-09-10T22:58:34.188921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:17.409553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602556+0000","last_change":"2024-09-10T22:56:27.955558+0000","last_active":"2024-09-10T22:58:34.602556+0000","last_peered":"2024-09-10T22:58:34.602556+0000","last_clean":"2024-09-10T22:58:34.602556+0000","last_became_active":"2024-09-10T22:56:27.955191+0000","last_became_peered":"2024-09-10T22:56:27.955191+0000","last_unstale":"2024-09-10T22:58:34.602556+0000","last_undegraded":"2024-09-10T22:58:34.602556+0000","last_fullsized":"2024-09-10T22:58:34.602556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:48.996693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464901+0000","last_change":"2024-09-10T22:56:18.918663+0000","last_active":"2024-09-10T22:58:34.464901+0000","last_peered":"2024-09-10T22:58:34.464901+0000","last_clean":"2024-09-10T22:58:34.464901+0000","last_became_active":"2024-09-10T22:56:18.912882+0000","last_became_peered":"2024-09-10T22:56:18.912882+0000","last_unstale":"2024-09-10T22:58:34.464901+0000","last_undegraded":"2024-09-10T22:58:34.464901+0000","last_fullsized":"2024-09-10T22:58:34.464901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.558836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:25.700231+0000","last_change":"2024-09-10T22:56:16.894232+0000","last_active":"2024-09-10T22:59:25.700231+0000","last_peered":"2024-09-10T22:59:25.700231+0000","last_clean":"2024-09-10T22:59:25.700231+0000","last_became_active":"2024-09-10T22:56:16.893542+0000","last_became_peered":"2024-09-10T22:56:16.893542+0000","last_unstale":"2024-09-10T22:59:25.700231+0000","last_undegraded":"2024-09-10T22:59:25.700231+0000","last_fullsized":"2024-09-10T22:59:25.700231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:44.220353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62814,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":277,"num_read_kb":200,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.604692+0000","last_change":"2024-09-10T22:58:34.604157+0000","last_active":"2024-09-10T22:58:34.604692+0000","last_peered":"2024-09-10T22:58:34.604692+0000","last_clean":"2024-09-10T22:58:34.604692+0000","last_became_active":"2024-09-10T22:58:34.603682+0000","last_became_peered":"2024-09-10T22:58:34.603682+0000","last_unstale":"2024-09-10T22:58:34.604692+0000","last_undegraded":"2024-09-10T22:58:34.604692+0000","last_fullsized":"2024-09-10T22:58:34.604692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:44.610556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587420+0000","last_change":"2024-09-10T22:56:18.915946+0000","last_active":"2024-09-10T22:58:34.587420+0000","last_peered":"2024-09-10T22:58:34.587420+0000","last_clean":"2024-09-10T22:58:34.587420+0000","last_became_active":"2024-09-10T22:56:18.911258+0000","last_became_peered":"2024-09-10T22:56:18.911258+0000","last_unstale":"2024-09-10T22:58:34.587420+0000","last_undegraded":"2024-09-10T22:58:34.587420+0000","last_fullsized":"2024-09-10T22:58:34.587420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602989+0000","last_change":"2024-09-10T22:56:17.605057+0000","last_active":"2024-09-10T22:58:34.602989+0000","last_peered":"2024-09-10T22:58:34.602989+0000","last_clean":"2024-09-10T22:58:34.602989+0000","last_became_active":"2024-09-10T22:56:17.604653+0000","last_became_peered":"2024-09-10T22:56:17.604653+0000","last_unstale":"2024-09-10T22:58:34.602989+0000","last_undegraded":"2024-09-10T22:58:34.602989+0000","last_fullsized":"2024-09-10T22:58:34.602989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:47.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.589489+0000","last_change":"2024-09-10T22:58:34.588867+0000","last_active":"2024-09-10T22:58:34.589489+0000","last_peered":"2024-09-10T22:58:34.589489+0000","last_clean":"2024-09-10T22:58:34.589489+0000","last_became_active":"2024-09-10T22:58:34.588410+0000","last_became_peered":"2024-09-10T22:58:34.588410+0000","last_unstale":"2024-09-10T22:58:34.589489+0000","last_undegraded":"2024-09-10T22:58:34.589489+0000","last_fullsized":"2024-09-10T22:58:34.589489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:24.765755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666891+0000","last_change":"2024-09-10T22:56:18.927791+0000","last_active":"2024-09-10T22:58:34.666891+0000","last_peered":"2024-09-10T22:58:34.666891+0000","last_clean":"2024-09-10T22:58:34.666891+0000","last_became_active":"2024-09-10T22:56:18.926129+0000","last_became_peered":"2024-09-10T22:56:18.926129+0000","last_unstale":"2024-09-10T22:58:34.666891+0000","last_undegraded":"2024-09-10T22:58:34.666891+0000","last_fullsized":"2024-09-10T22:58:34.666891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:42.462350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185860+0000","last_change":"2024-09-10T22:56:16.900187+0000","last_active":"2024-09-10T22:58:34.185860+0000","last_peered":"2024-09-10T22:58:34.185860+0000","last_clean":"2024-09-10T22:58:34.185860+0000","last_became_active":"2024-09-10T22:56:16.899795+0000","last_became_peered":"2024-09-10T22:56:16.899795+0000","last_unstale":"2024-09-10T22:58:34.185860+0000","last_undegraded":"2024-09-10T22:58:34.185860+0000","last_fullsized":"2024-09-10T22:58:34.185860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:38.126345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184137+0000","last_change":"2024-09-10T22:56:27.956023+0000","last_active":"2024-09-10T22:58:34.184137+0000","last_peered":"2024-09-10T22:58:34.184137+0000","last_clean":"2024-09-10T22:58:34.184137+0000","last_became_active":"2024-09-10T22:56:27.953793+0000","last_became_peered":"2024-09-10T22:56:27.953793+0000","last_unstale":"2024-09-10T22:58:34.184137+0000","last_undegraded":"2024-09-10T22:58:34.184137+0000","last_fullsized":"2024-09-10T22:58:34.184137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:30.761943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.921888+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.917288+0000","last_became_peered":"2024-09-10T22:56:18.917288+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:05.679055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:24.698215+0000","last_change":"2024-09-10T22:56:16.890056+0000","last_active":"2024-09-10T22:59:24.698215+0000","last_peered":"2024-09-10T22:59:24.698215+0000","last_clean":"2024-09-10T22:59:24.698215+0000","last_became_active":"2024-09-10T22:56:16.889495+0000","last_became_peered":"2024-09-10T22:56:16.889495+0000","last_unstale":"2024-09-10T22:59:24.698215+0000","last_undegraded":"2024-09-10T22:59:24.698215+0000","last_fullsized":"2024-09-10T22:59:24.698215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:27.789062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462520+0000","last_change":"2024-09-10T22:56:27.964527+0000","last_active":"2024-09-10T22:58:34.462520+0000","last_peered":"2024-09-10T22:58:34.462520+0000","last_clean":"2024-09-10T22:58:34.462520+0000","last_became_active":"2024-09-10T22:56:27.964172+0000","last_became_peered":"2024-09-10T22:56:27.964172+0000","last_unstale":"2024-09-10T22:58:34.462520+0000","last_undegraded":"2024-09-10T22:58:34.462520+0000","last_fullsized":"2024-09-10T22:58:34.462520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:05.130649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666395+0000","last_change":"2024-09-10T22:56:27.959389+0000","last_active":"2024-09-10T22:58:34.666395+0000","last_peered":"2024-09-10T22:58:34.666395+0000","last_clean":"2024-09-10T22:58:34.666395+0000","last_became_active":"2024-09-10T22:56:27.957778+0000","last_became_peered":"2024-09-10T22:56:27.957778+0000","last_unstale":"2024-09-10T22:58:34.666395+0000","last_undegraded":"2024-09-10T22:58:34.666395+0000","last_fullsized":"2024-09-10T22:58:34.666395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:56.014474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184232+0000","last_change":"2024-09-10T22:56:18.913635+0000","last_active":"2024-09-10T22:58:34.184232+0000","last_peered":"2024-09-10T22:58:34.184232+0000","last_clean":"2024-09-10T22:58:34.184232+0000","last_became_active":"2024-09-10T22:56:18.913283+0000","last_became_peered":"2024-09-10T22:56:18.913283+0000","last_unstale":"2024-09-10T22:58:34.184232+0000","last_undegraded":"2024-09-10T22:58:34.184232+0000","last_fullsized":"2024-09-10T22:58:34.184232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:35.509470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602683+0000","last_change":"2024-09-10T22:56:16.896282+0000","last_active":"2024-09-10T22:58:34.602683+0000","last_peered":"2024-09-10T22:58:34.602683+0000","last_clean":"2024-09-10T22:58:34.602683+0000","last_became_active":"2024-09-10T22:56:16.895966+0000","last_became_peered":"2024-09-10T22:56:16.895966+0000","last_unstale":"2024-09-10T22:58:34.602683+0000","last_undegraded":"2024-09-10T22:58:34.602683+0000","last_fullsized":"2024-09-10T22:58:34.602683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:54.460409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186357+0000","last_change":"2024-09-10T22:56:27.955971+0000","last_active":"2024-09-10T22:58:34.186357+0000","last_peered":"2024-09-10T22:58:34.186357+0000","last_clean":"2024-09-10T22:58:34.186357+0000","last_became_active":"2024-09-10T22:56:27.955589+0000","last_became_peered":"2024-09-10T22:56:27.955589+0000","last_unstale":"2024-09-10T22:58:34.186357+0000","last_undegraded":"2024-09-10T22:58:34.186357+0000","last_fullsized":"2024-09-10T22:58:34.186357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:50.015159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462533+0000","last_change":"2024-09-10T22:56:18.913569+0000","last_active":"2024-09-10T22:58:34.462533+0000","last_peered":"2024-09-10T22:58:34.462533+0000","last_clean":"2024-09-10T22:58:34.462533+0000","last_became_active":"2024-09-10T22:56:18.912410+0000","last_became_peered":"2024-09-10T22:56:18.912410+0000","last_unstale":"2024-09-10T22:58:34.462533+0000","last_undegraded":"2024-09-10T22:58:34.462533+0000","last_fullsized":"2024-09-10T22:58:34.462533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:20.670270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186324+0000","last_change":"2024-09-10T22:56:16.889994+0000","last_active":"2024-09-10T22:58:34.186324+0000","last_peered":"2024-09-10T22:58:34.186324+0000","last_clean":"2024-09-10T22:58:34.186324+0000","last_became_active":"2024-09-10T22:56:16.889686+0000","last_became_peered":"2024-09-10T22:56:16.889686+0000","last_unstale":"2024-09-10T22:58:34.186324+0000","last_undegraded":"2024-09-10T22:58:34.186324+0000","last_fullsized":"2024-09-10T22:58:34.186324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:13.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189561+0000","last_change":"2024-09-10T22:56:16.891226+0000","last_active":"2024-09-10T22:58:34.189561+0000","last_peered":"2024-09-10T22:58:34.189561+0000","last_clean":"2024-09-10T22:58:34.189561+0000","last_became_active":"2024-09-10T22:56:16.890855+0000","last_became_peered":"2024-09-10T22:56:16.890855+0000","last_unstale":"2024-09-10T22:58:34.189561+0000","last_undegraded":"2024-09-10T22:58:34.189561+0000","last_fullsized":"2024-09-10T22:58:34.189561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:28.806919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602398+0000","last_change":"2024-09-10T22:56:18.923411+0000","last_active":"2024-09-10T22:58:34.602398+0000","last_peered":"2024-09-10T22:58:34.602398+0000","last_clean":"2024-09-10T22:58:34.602398+0000","last_became_active":"2024-09-10T22:56:18.921233+0000","last_became_peered":"2024-09-10T22:56:18.921233+0000","last_unstale":"2024-09-10T22:58:34.602398+0000","last_undegraded":"2024-09-10T22:58:34.602398+0000","last_fullsized":"2024-09-10T22:58:34.602398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:50.335996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188728+0000","last_change":"2024-09-10T22:56:27.960352+0000","last_active":"2024-09-10T22:58:34.188728+0000","last_peered":"2024-09-10T22:58:34.188728+0000","last_clean":"2024-09-10T22:58:34.188728+0000","last_became_active":"2024-09-10T22:56:27.959958+0000","last_became_peered":"2024-09-10T22:56:27.959958+0000","last_unstale":"2024-09-10T22:58:34.188728+0000","last_undegraded":"2024-09-10T22:58:34.188728+0000","last_fullsized":"2024-09-10T22:58:34.188728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:34.486737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":202,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:59:26.702550+0000","last_change":"2024-09-10T22:57:33.496995+0000","last_active":"2024-09-10T22:59:26.702550+0000","last_peered":"2024-09-10T22:59:26.702550+0000","last_clean":"2024-09-10T22:59:26.702550+0000","last_became_active":"2024-09-10T22:57:33.491192+0000","last_became_peered":"2024-09-10T22:57:33.491192+0000","last_unstale":"2024-09-10T22:59:26.702550+0000","last_undegraded":"2024-09-10T22:59:26.702550+0000","last_fullsized":"2024-09-10T22:59:26.702550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:15.859533+0000","last_clean_scrub_stamp":"2024-09-10T22:56:15.859533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:03.627940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464546+0000","last_change":"2024-09-10T22:56:18.912554+0000","last_active":"2024-09-10T22:58:34.464546+0000","last_peered":"2024-09-10T22:58:34.464546+0000","last_clean":"2024-09-10T22:58:34.464546+0000","last_became_active":"2024-09-10T22:56:18.912186+0000","last_became_peered":"2024-09-10T22:56:18.912186+0000","last_unstale":"2024-09-10T22:58:34.464546+0000","last_undegraded":"2024-09-10T22:58:34.464546+0000","last_fullsized":"2024-09-10T22:58:34.464546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:16.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189292+0000","last_change":"2024-09-10T22:56:27.953157+0000","last_active":"2024-09-10T22:58:34.189292+0000","last_peered":"2024-09-10T22:58:34.189292+0000","last_clean":"2024-09-10T22:58:34.189292+0000","last_became_active":"2024-09-10T22:56:27.952549+0000","last_became_peered":"2024-09-10T22:56:27.952549+0000","last_unstale":"2024-09-10T22:58:34.189292+0000","last_undegraded":"2024-09-10T22:58:34.189292+0000","last_fullsized":"2024-09-10T22:58:34.189292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:26.929198+0000","last_clean_scrub_stamp":"2024-09-10T22:56:26.929198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:10.200463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185970+0000","last_change":"2024-09-10T22:56:18.914449+0000","last_active":"2024-09-10T22:58:34.185970+0000","last_peered":"2024-09-10T22:58:34.185970+0000","last_clean":"2024-09-10T22:58:34.185970+0000","last_became_active":"2024-09-10T22:56:18.913843+0000","last_became_peered":"2024-09-10T22:56:18.913843+0000","last_unstale":"2024-09-10T22:58:34.185970+0000","last_undegraded":"2024-09-10T22:58:34.185970+0000","last_fullsized":"2024-09-10T22:58:34.185970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:43.230656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189242+0000","last_change":"2024-09-10T22:56:18.923832+0000","last_active":"2024-09-10T22:58:34.189242+0000","last_peered":"2024-09-10T22:58:34.189242+0000","last_clean":"2024-09-10T22:58:34.189242+0000","last_became_active":"2024-09-10T22:56:18.918635+0000","last_became_peered":"2024-09-10T22:56:18.918635+0000","last_unstale":"2024-09-10T22:58:34.189242+0000","last_undegraded":"2024-09-10T22:58:34.189242+0000","last_fullsized":"2024-09-10T22:58:34.189242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:53.372581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188923+0000","last_change":"2024-09-10T22:56:18.918621+0000","last_active":"2024-09-10T22:58:34.188923+0000","last_peered":"2024-09-10T22:58:34.188923+0000","last_clean":"2024-09-10T22:58:34.188923+0000","last_became_active":"2024-09-10T22:56:18.916910+0000","last_became_peered":"2024-09-10T22:56:18.916910+0000","last_unstale":"2024-09-10T22:58:34.188923+0000","last_undegraded":"2024-09-10T22:58:34.188923+0000","last_fullsized":"2024-09-10T22:58:34.188923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:20.802956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666291+0000","last_change":"2024-09-10T22:56:18.930047+0000","last_active":"2024-09-10T22:58:34.666291+0000","last_peered":"2024-09-10T22:58:34.666291+0000","last_clean":"2024-09-10T22:58:34.666291+0000","last_became_active":"2024-09-10T22:56:18.928625+0000","last_became_peered":"2024-09-10T22:56:18.928625+0000","last_unstale":"2024-09-10T22:58:34.666291+0000","last_undegraded":"2024-09-10T22:58:34.666291+0000","last_fullsized":"2024-09-10T22:58:34.666291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:11.871680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186557+0000","last_change":"2024-09-10T22:56:18.919829+0000","last_active":"2024-09-10T22:58:34.186557+0000","last_peered":"2024-09-10T22:58:34.186557+0000","last_clean":"2024-09-10T22:58:34.186557+0000","last_became_active":"2024-09-10T22:56:18.919488+0000","last_became_peered":"2024-09-10T22:56:18.919488+0000","last_unstale":"2024-09-10T22:58:34.186557+0000","last_undegraded":"2024-09-10T22:58:34.186557+0000","last_fullsized":"2024-09-10T22:58:34.186557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:58.684037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186283+0000","last_change":"2024-09-10T22:56:18.919267+0000","last_active":"2024-09-10T22:58:34.186283+0000","last_peered":"2024-09-10T22:58:34.186283+0000","last_clean":"2024-09-10T22:58:34.186283+0000","last_became_active":"2024-09-10T22:56:18.917765+0000","last_became_peered":"2024-09-10T22:56:18.917765+0000","last_unstale":"2024-09-10T22:58:34.186283+0000","last_undegraded":"2024-09-10T22:58:34.186283+0000","last_fullsized":"2024-09-10T22:58:34.186283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:57.571830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189258+0000","last_change":"2024-09-10T22:56:18.916295+0000","last_active":"2024-09-10T22:58:34.189258+0000","last_peered":"2024-09-10T22:58:34.189258+0000","last_clean":"2024-09-10T22:58:34.189258+0000","last_became_active":"2024-09-10T22:56:18.915799+0000","last_became_peered":"2024-09-10T22:56:18.915799+0000","last_unstale":"2024-09-10T22:58:34.189258+0000","last_undegraded":"2024-09-10T22:58:34.189258+0000","last_fullsized":"2024-09-10T22:58:34.189258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:27.386764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462967+0000","last_change":"2024-09-10T22:56:18.924459+0000","last_active":"2024-09-10T22:58:34.462967+0000","last_peered":"2024-09-10T22:58:34.462967+0000","last_clean":"2024-09-10T22:58:34.462967+0000","last_became_active":"2024-09-10T22:56:18.924110+0000","last_became_peered":"2024-09-10T22:56:18.924110+0000","last_unstale":"2024-09-10T22:58:34.462967+0000","last_undegraded":"2024-09-10T22:58:34.462967+0000","last_fullsized":"2024-09-10T22:58:34.462967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:51.325105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.602043+0000","last_change":"2024-09-10T22:56:18.916857+0000","last_active":"2024-09-10T22:58:34.602043+0000","last_peered":"2024-09-10T22:58:34.602043+0000","last_clean":"2024-09-10T22:58:34.602043+0000","last_became_active":"2024-09-10T22:56:18.916525+0000","last_became_peered":"2024-09-10T22:56:18.916525+0000","last_unstale":"2024-09-10T22:58:34.602043+0000","last_undegraded":"2024-09-10T22:58:34.602043+0000","last_fullsized":"2024-09-10T22:58:34.602043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.166794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462583+0000","last_change":"2024-09-10T22:56:18.923305+0000","last_active":"2024-09-10T22:58:34.462583+0000","last_peered":"2024-09-10T22:58:34.462583+0000","last_clean":"2024-09-10T22:58:34.462583+0000","last_became_active":"2024-09-10T22:56:18.922256+0000","last_became_peered":"2024-09-10T22:56:18.922256+0000","last_unstale":"2024-09-10T22:58:34.462583+0000","last_undegraded":"2024-09-10T22:58:34.462583+0000","last_fullsized":"2024-09-10T22:58:34.462583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:04.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587859+0000","last_change":"2024-09-10T22:56:18.922520+0000","last_active":"2024-09-10T22:58:34.587859+0000","last_peered":"2024-09-10T22:58:34.587859+0000","last_clean":"2024-09-10T22:58:34.587859+0000","last_became_active":"2024-09-10T22:56:18.920102+0000","last_became_peered":"2024-09-10T22:56:18.920102+0000","last_unstale":"2024-09-10T22:58:34.587859+0000","last_undegraded":"2024-09-10T22:58:34.587859+0000","last_fullsized":"2024-09-10T22:58:34.587859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.073547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186971+0000","last_change":"2024-09-10T22:56:18.917332+0000","last_active":"2024-09-10T22:58:34.186971+0000","last_peered":"2024-09-10T22:58:34.186971+0000","last_clean":"2024-09-10T22:58:34.186971+0000","last_became_active":"2024-09-10T22:56:18.916997+0000","last_became_peered":"2024-09-10T22:56:18.916997+0000","last_unstale":"2024-09-10T22:58:34.186971+0000","last_undegraded":"2024-09-10T22:58:34.186971+0000","last_fullsized":"2024-09-10T22:58:34.186971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:39.776124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186021+0000","last_change":"2024-09-10T22:56:18.921813+0000","last_active":"2024-09-10T22:58:34.186021+0000","last_peered":"2024-09-10T22:58:34.186021+0000","last_clean":"2024-09-10T22:58:34.186021+0000","last_became_active":"2024-09-10T22:56:18.921417+0000","last_became_peered":"2024-09-10T22:56:18.921417+0000","last_unstale":"2024-09-10T22:58:34.186021+0000","last_undegraded":"2024-09-10T22:58:34.186021+0000","last_fullsized":"2024-09-10T22:58:34.186021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:53.693512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189081+0000","last_change":"2024-09-10T22:56:18.924799+0000","last_active":"2024-09-10T22:58:34.189081+0000","last_peered":"2024-09-10T22:58:34.189081+0000","last_clean":"2024-09-10T22:58:34.189081+0000","last_became_active":"2024-09-10T22:56:18.924165+0000","last_became_peered":"2024-09-10T22:56:18.924165+0000","last_unstale":"2024-09-10T22:58:34.189081+0000","last_undegraded":"2024-09-10T22:58:34.189081+0000","last_fullsized":"2024-09-10T22:58:34.189081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:44.717879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.184788+0000","last_change":"2024-09-10T22:56:18.914773+0000","last_active":"2024-09-10T22:58:34.184788+0000","last_peered":"2024-09-10T22:58:34.184788+0000","last_clean":"2024-09-10T22:58:34.184788+0000","last_became_active":"2024-09-10T22:56:18.911633+0000","last_became_peered":"2024-09-10T22:56:18.911633+0000","last_unstale":"2024-09-10T22:58:34.184788+0000","last_undegraded":"2024-09-10T22:58:34.184788+0000","last_fullsized":"2024-09-10T22:58:34.184788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:43.871747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186656+0000","last_change":"2024-09-10T22:56:18.928316+0000","last_active":"2024-09-10T22:58:34.186656+0000","last_peered":"2024-09-10T22:58:34.186656+0000","last_clean":"2024-09-10T22:58:34.186656+0000","last_became_active":"2024-09-10T22:56:18.927683+0000","last_became_peered":"2024-09-10T22:56:18.927683+0000","last_unstale":"2024-09-10T22:58:34.186656+0000","last_undegraded":"2024-09-10T22:58:34.186656+0000","last_fullsized":"2024-09-10T22:58:34.186656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:53.143958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587130+0000","last_change":"2024-09-10T22:56:18.915754+0000","last_active":"2024-09-10T22:58:34.587130+0000","last_peered":"2024-09-10T22:58:34.587130+0000","last_clean":"2024-09-10T22:58:34.587130+0000","last_became_active":"2024-09-10T22:56:18.911265+0000","last_became_peered":"2024-09-10T22:56:18.911265+0000","last_unstale":"2024-09-10T22:58:34.587130+0000","last_undegraded":"2024-09-10T22:58:34.587130+0000","last_fullsized":"2024-09-10T22:58:34.587130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:53.058642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666125+0000","last_change":"2024-09-10T22:56:18.930114+0000","last_active":"2024-09-10T22:58:34.666125+0000","last_peered":"2024-09-10T22:58:34.666125+0000","last_clean":"2024-09-10T22:58:34.666125+0000","last_became_active":"2024-09-10T22:56:18.929745+0000","last_became_peered":"2024-09-10T22:56:18.929745+0000","last_unstale":"2024-09-10T22:58:34.666125+0000","last_undegraded":"2024-09-10T22:58:34.666125+0000","last_fullsized":"2024-09-10T22:58:34.666125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:49.174204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666081+0000","last_change":"2024-09-10T22:56:18.921254+0000","last_active":"2024-09-10T22:58:34.666081+0000","last_peered":"2024-09-10T22:58:34.666081+0000","last_clean":"2024-09-10T22:58:34.666081+0000","last_became_active":"2024-09-10T22:56:18.920828+0000","last_became_peered":"2024-09-10T22:56:18.920828+0000","last_unstale":"2024-09-10T22:58:34.666081+0000","last_undegraded":"2024-09-10T22:58:34.666081+0000","last_fullsized":"2024-09-10T22:58:34.666081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:09.978952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186817+0000","last_change":"2024-09-10T22:56:18.915127+0000","last_active":"2024-09-10T22:58:34.186817+0000","last_peered":"2024-09-10T22:58:34.186817+0000","last_clean":"2024-09-10T22:58:34.186817+0000","last_became_active":"2024-09-10T22:56:18.914754+0000","last_became_peered":"2024-09-10T22:56:18.914754+0000","last_unstale":"2024-09-10T22:58:34.186817+0000","last_undegraded":"2024-09-10T22:58:34.186817+0000","last_fullsized":"2024-09-10T22:58:34.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.330131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.188049+0000","last_change":"2024-09-10T22:56:18.908599+0000","last_active":"2024-09-10T22:58:34.188049+0000","last_peered":"2024-09-10T22:58:34.188049+0000","last_clean":"2024-09-10T22:58:34.188049+0000","last_became_active":"2024-09-10T22:56:18.908184+0000","last_became_peered":"2024-09-10T22:56:18.908184+0000","last_unstale":"2024-09-10T22:58:34.188049+0000","last_undegraded":"2024-09-10T22:58:34.188049+0000","last_fullsized":"2024-09-10T22:58:34.188049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:20.250033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186351+0000","last_change":"2024-09-10T22:56:18.924457+0000","last_active":"2024-09-10T22:58:34.186351+0000","last_peered":"2024-09-10T22:58:34.186351+0000","last_clean":"2024-09-10T22:58:34.186351+0000","last_became_active":"2024-09-10T22:56:18.924154+0000","last_became_peered":"2024-09-10T22:56:18.924154+0000","last_unstale":"2024-09-10T22:58:34.186351+0000","last_undegraded":"2024-09-10T22:58:34.186351+0000","last_fullsized":"2024-09-10T22:58:34.186351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:00.193455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189733+0000","last_change":"2024-09-10T22:56:18.923948+0000","last_active":"2024-09-10T22:58:34.189733+0000","last_peered":"2024-09-10T22:58:34.189733+0000","last_clean":"2024-09-10T22:58:34.189733+0000","last_became_active":"2024-09-10T22:56:18.923453+0000","last_became_peered":"2024-09-10T22:56:18.923453+0000","last_unstale":"2024-09-10T22:58:34.189733+0000","last_undegraded":"2024-09-10T22:58:34.189733+0000","last_fullsized":"2024-09-10T22:58:34.189733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.195477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.587752+0000","last_change":"2024-09-10T22:56:18.920552+0000","last_active":"2024-09-10T22:58:34.587752+0000","last_peered":"2024-09-10T22:58:34.587752+0000","last_clean":"2024-09-10T22:58:34.587752+0000","last_became_active":"2024-09-10T22:56:18.919523+0000","last_became_peered":"2024-09-10T22:56:18.919523+0000","last_unstale":"2024-09-10T22:58:34.587752+0000","last_undegraded":"2024-09-10T22:58:34.587752+0000","last_fullsized":"2024-09-10T22:58:34.587752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:33.781396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.462478+0000","last_change":"2024-09-10T22:56:18.920007+0000","last_active":"2024-09-10T22:58:34.462478+0000","last_peered":"2024-09-10T22:58:34.462478+0000","last_clean":"2024-09-10T22:58:34.462478+0000","last_became_active":"2024-09-10T22:56:18.919081+0000","last_became_peered":"2024-09-10T22:56:18.919081+0000","last_unstale":"2024-09-10T22:58:34.462478+0000","last_undegraded":"2024-09-10T22:58:34.462478+0000","last_fullsized":"2024-09-10T22:58:34.462478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:35.578352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.185517+0000","last_change":"2024-09-10T22:56:18.919518+0000","last_active":"2024-09-10T22:58:34.185517+0000","last_peered":"2024-09-10T22:58:34.185517+0000","last_clean":"2024-09-10T22:58:34.185517+0000","last_became_active":"2024-09-10T22:56:18.916491+0000","last_became_peered":"2024-09-10T22:56:18.916491+0000","last_unstale":"2024-09-10T22:58:34.185517+0000","last_undegraded":"2024-09-10T22:58:34.185517+0000","last_fullsized":"2024-09-10T22:58:34.185517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:56.871230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.189448+0000","last_change":"2024-09-10T22:57:33.495873+0000","last_active":"2024-09-10T22:58:34.189448+0000","last_peered":"2024-09-10T22:58:34.189448+0000","last_clean":"2024-09-10T22:58:34.189448+0000","last_became_active":"2024-09-10T22:57:33.495180+0000","last_became_peered":"2024-09-10T22:57:33.495180+0000","last_unstale":"2024-09-10T22:58:34.189448+0000","last_undegraded":"2024-09-10T22:58:34.189448+0000","last_fullsized":"2024-09-10T22:58:34.189448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:48.082520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.464386+0000","last_change":"2024-09-10T22:56:18.914722+0000","last_active":"2024-09-10T22:58:34.464386+0000","last_peered":"2024-09-10T22:58:34.464386+0000","last_clean":"2024-09-10T22:58:34.464386+0000","last_became_active":"2024-09-10T22:56:18.914381+0000","last_became_peered":"2024-09-10T22:56:18.914381+0000","last_unstale":"2024-09-10T22:58:34.464386+0000","last_undegraded":"2024-09-10T22:58:34.464386+0000","last_fullsized":"2024-09-10T22:58:34.464386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:07.265570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.187920+0000","last_change":"2024-09-10T22:56:18.917258+0000","last_active":"2024-09-10T22:58:34.187920+0000","last_peered":"2024-09-10T22:58:34.187920+0000","last_clean":"2024-09-10T22:58:34.187920+0000","last_became_active":"2024-09-10T22:56:18.915407+0000","last_became_peered":"2024-09-10T22:56:18.915407+0000","last_unstale":"2024-09-10T22:58:34.187920+0000","last_undegraded":"2024-09-10T22:58:34.187920+0000","last_fullsized":"2024-09-10T22:58:34.187920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:47.939866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666406+0000","last_change":"2024-09-10T22:56:18.930763+0000","last_active":"2024-09-10T22:58:34.666406+0000","last_peered":"2024-09-10T22:58:34.666406+0000","last_clean":"2024-09-10T22:58:34.666406+0000","last_became_active":"2024-09-10T22:56:18.930296+0000","last_became_peered":"2024-09-10T22:56:18.930296+0000","last_unstale":"2024-09-10T22:58:34.666406+0000","last_undegraded":"2024-09-10T22:58:34.666406+0000","last_fullsized":"2024-09-10T22:58:34.666406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:41.629833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.666421+0000","last_change":"2024-09-10T22:56:18.922051+0000","last_active":"2024-09-10T22:58:34.666421+0000","last_peered":"2024-09-10T22:58:34.666421+0000","last_clean":"2024-09-10T22:58:34.666421+0000","last_became_active":"2024-09-10T22:56:18.918534+0000","last_became_peered":"2024-09-10T22:56:18.918534+0000","last_unstale":"2024-09-10T22:58:34.666421+0000","last_undegraded":"2024-09-10T22:58:34.666421+0000","last_fullsized":"2024-09-10T22:58:34.666421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:29.580230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T22:58:34.186517+0000","last_change":"2024-09-10T22:56:18.929214+0000","last_active":"2024-09-10T22:58:34.186517+0000","last_peered":"2024-09-10T22:58:34.186517+0000","last_clean":"2024-09-10T22:58:34.186517+0000","last_became_active":"2024-09-10T22:56:18.928880+0000","last_became_peered":"2024-09-10T22:56:18.928880+0000","last_unstale":"2024-09-10T22:58:34.186517+0000","last_undegraded":"2024-09-10T22:58:34.186517+0000","last_fullsized":"2024-09-10T22:58:34.186517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:56:17.875402+0000","last_clean_scrub_stamp":"2024-09-10T22:56:17.875402+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:33.897335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":854934,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1173,"num_read_kb":792,"num_write":409,"num_write_kb":2218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":2573430,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281969,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612662,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518482,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808891,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1179648,"data_stored":1562721,"data_compressed":83571,"data_compressed_allocated":110592,"data_compressed_original":1040384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":915918,"data_compressed":48912,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168649,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":911630,"data_compressed":48912,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397581,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1115562,"data_compressed":65480,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561108,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":513782,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757338,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":1504433,"data_compressed":83571,"data_compressed_allocated":110592,"data_compressed_original":1040384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920864,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":519037,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117093,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":1300968,"data_compressed":67003,"data_compressed_allocated":90112,"data_compressed_original":843776,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378733,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":581436,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542258,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":582467,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705784,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":999424,"data_stored":515170,"data_compressed":29623,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34228,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35857,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":396277,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":458661,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:59:32.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:32.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:32 smithi092 ceph-mon[34223]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:59:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:33.513 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:33 smithi067 ceph-mon[30939]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.624+0000 7ff204800640 1 -- 172.21.15.67:0/2422522506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc10b320 msgr2=0x7ff1fc111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.624+0000 7ff204800640 1 --2- 172.21.15.67:0/2422522506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc10b320 0x7ff1fc111b20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009ca0 tx=0x7ff1f802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 -- 172.21.15.67:0/2422522506 shutdown_connections 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 --2- 172.21.15.67:0/2422522506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc10b320 0x7ff1fc111b20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 --2- 172.21.15.67:0/2422522506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 0x7ff1fc10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 --2- 172.21.15.67:0/2422522506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc104f90 0x7ff1fc105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 -- 172.21.15.67:0/2422522506 >> 172.21.15.67:0/2422522506 conn(0x7ff1fc100a00 msgr2=0x7ff1fc102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 -- 172.21.15.67:0/2422522506 shutdown_connections 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.625+0000 7ff204800640 1 -- 172.21.15.67:0/2422522506 wait complete. 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 Processor -- start 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 -- start start 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 0x7ff1fc19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc10b320 0x7ff1fc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114e50 con 0x7ff1fc104f90 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114cd0 con 0x7ff1fc10a9a0 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff204800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114fd0 con 0x7ff1fc10b320 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff201e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff201e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44568/0 (socket says 172.21.15.67:44568) 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.626+0000 7ff201e00640 1 -- 172.21.15.67:0/198373382 learned_addr learned my addr 172.21.15.67:0/198373382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff202800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc10b320 0x7ff1fc1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201400640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 0x7ff1fc19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 -- 172.21.15.67:0/198373382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc10b320 msgr2=0x7ff1fc1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc10b320 0x7ff1fc1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 -- 172.21.15.67:0/198373382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 msgr2=0x7ff1fc19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 0x7ff1fc19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 -- 172.21.15.67:0/198373382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1a1350 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff201e00640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000c920 tx=0x7ff1f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.627+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0007c70 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.628+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc1a1640 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.628+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f0007e10 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.628+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00186f0 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.628+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a7d00 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.629+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc105370 con 0x7ff1fc104f90 2024-09-10T22:59:33.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:33 smithi145 ceph-mon[34397]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.631+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7ff1f000f450 con 0x7ff1fc104f90 2024-09-10T22:59:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.632+0000 7ff1f6a00640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 0x7ff1e40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.633+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff1f00e66e0 con 0x7ff1fc104f90 2024-09-10T22:59:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.633+0000 7ff201400640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 0x7ff1e40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:33.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.633+0000 7ff201400640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 0x7ff1e40b7880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc104360 tx=0x7ff1ec00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:33.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.634+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1f0014030 con 0x7ff1fc104f90 2024-09-10T22:59:33.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:33 smithi092 ceph-mon[34223]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.870+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1fc02e5c0 con 0x7ff1fc104f90 2024-09-10T22:59:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.871+0000 7ff1f6a00640 1 -- 172.21.15.67:0/198373382 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11903 (secure 0 0 0) 0x7ff1f00ad020 con 0x7ff1fc104f90 2024-09-10T22:59:33.872 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:33.872 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":27,"btime":"2024-09-10T22:59:24:334283+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:59:23.805517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T22:59:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 27 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 msgr2=0x7ff1e40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 0x7ff1e40b7880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc104360 tx=0x7ff1ec00b390 comp rx=0 tx=0).stop 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 msgr2=0x7ff1fc19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000c920 tx=0x7ff1f000cde0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 shutdown_connections 2024-09-10T22:59:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1e40b53c0 0x7ff1e40b7880 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1fc10b320 0x7ff1fc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1fc10a9a0 0x7ff1fc19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 --2- 172.21.15.67:0/198373382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1fc104f90 0x7ff1fc19b2a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 >> 172.21.15.67:0/198373382 conn(0x7ff1fc100a00 msgr2=0x7ff1fc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 shutdown_connections 2024-09-10T22:59:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:33.873+0000 7ff204800640 1 -- 172.21.15.67:0/198373382 wait complete. 2024-09-10T22:59:34.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:34 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/198373382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:34.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:34.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:34 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/198373382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:34.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:34 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/198373382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:35.061 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:35.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:35 smithi067 ceph-mon[30939]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:35.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:35 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 -- 172.21.15.67:0/1035869827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 msgr2=0x7fd8ac10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 --2- 172.21.15.67:0/1035869827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac10f3d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009ca0 tx=0x7fd8a402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 -- 172.21.15.67:0/1035869827 shutdown_connections 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 --2- 172.21.15.67:0/1035869827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac10f3d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 --2- 172.21.15.67:0/1035869827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 --2- 172.21.15.67:0/1035869827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 -- 172.21.15.67:0/1035869827 >> 172.21.15.67:0/1035869827 conn(0x7fd8ac0fd5b0 msgr2=0x7fd8ac0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.691+0000 7fd8b2800640 1 -- 172.21.15.67:0/1035869827 shutdown_connections 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.692+0000 7fd8b2800640 1 -- 172.21.15.67:0/1035869827 wait complete. 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.693+0000 7fd8b2800640 1 Processor -- start 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.693+0000 7fd8b2800640 1 -- start start 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.693+0000 7fd8b2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b2800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112ae0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b2800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112960 con 0x7fd8ac104430 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112c60 con 0x7fd8ac103b10 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44584/0 (socket says 172.21.15.67:44584) 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:52194/0 (socket says 172.21.15.67:52194) 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b1e00640 1 -- 172.21.15.67:0/2251354205 learned_addr learned my addr 172.21.15.67:0/2251354205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.694+0000 7fd8b1400640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 -- 172.21.15.67:0/2251354205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 msgr2=0x7fd8ac198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 -- 172.21.15.67:0/2251354205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 msgr2=0x7fd8ac199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 -- 172.21.15.67:0/2251354205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac19efc0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1400640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b1e00640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4005e30 tx=0x7fd8a4003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4032c10 con 0x7fd8ac10cf70 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8a4032db0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4030ab0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.695+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac19f250 con 0x7fd8ac10cf70 2024-09-10T22:59:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.696+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1a5970 con 0x7fd8ac10cf70 2024-09-10T22:59:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.697+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac103ef0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.699+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7fd8a4030c50 con 0x7fd8ac10cf70 2024-09-10T22:59:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.702+0000 7fd8aa000640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 0x7fd8940b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:35.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.702+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd8a4107470 con 0x7fd8ac10cf70 2024-09-10T22:59:35.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.702+0000 7fd8b1400640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 0x7fd8940b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:35.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.703+0000 7fd8b1400640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 0x7fd8940b79a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009fc0 tx=0x7fd8a0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:35.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.709+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd8a40cdd50 con 0x7fd8ac10cf70 2024-09-10T22:59:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:35 smithi092 ceph-mon[34223]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:35 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:35.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:35 smithi145 ceph-mon[34397]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-10T22:59:35.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:35 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.951+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8ac1a5ca0 con 0x7fd8ac10cf70 2024-09-10T22:59:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.952+0000 7fd8aa000640 1 -- 172.21.15.67:0/2251354205 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11903 (secure 0 0 0) 0x7fd8a40cd650 con 0x7fd8ac10cf70 2024-09-10T22:59:35.954 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:35.954 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":27,"btime":"2024-09-10T22:59:24:334283+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:59:23.805517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T22:59:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 27 2024-09-10T22:59:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 msgr2=0x7fd8940b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 0x7fd8940b79a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009fc0 tx=0x7fd8a0005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 msgr2=0x7fd8ac19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:35.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4005e30 tx=0x7fd8a4003f30 comp rx=0 tx=0).stop 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 shutdown_connections 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd8940b54e0 0x7fd8940b79a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8ac10cf70 0x7fd8ac19e7f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac104430 0x7fd8ac199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.954+0000 7fd8b2800640 1 --2- 172.21.15.67:0/2251354205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ac103b10 0x7fd8ac198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.955+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 >> 172.21.15.67:0/2251354205 conn(0x7fd8ac0fd5b0 msgr2=0x7fd8ac10f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.955+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 shutdown_connections 2024-09-10T22:59:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:35.955+0000 7fd8b2800640 1 -- 172.21.15.67:0/2251354205 wait complete. 2024-09-10T22:59:36.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:36.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:36.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:59:36.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:36 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2251354205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:36.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:59:36.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:36.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:36.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:59:36.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:36 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2251354205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:36.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:36.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:36.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T22:59:36.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:36 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2251354205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 -- 172.21.15.67:0/2331761633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 msgr2=0x7f340010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 --2- 172.21.15.67:0/2331761633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f340010ae10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009900 tx=0x7f33ec02a400 comp rx=0 tx=0).stop 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 -- 172.21.15.67:0/2331761633 shutdown_connections 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 --2- 172.21.15.67:0/2331761633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f340010b350 0x7f3400111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 --2- 172.21.15.67:0/2331761633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f340010ae10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 --2- 172.21.15.67:0/2331761633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 0x7f34001053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 -- 172.21.15.67:0/2331761633 >> 172.21.15.67:0/2331761633 conn(0x7f34001009f0 msgr2=0x7f3400102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 -- 172.21.15.67:0/2331761633 shutdown_connections 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.704+0000 7f3408400640 1 -- 172.21.15.67:0/2331761633 wait complete. 2024-09-10T22:59:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.705+0000 7f3408400640 1 Processor -- start 2024-09-10T22:59:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.705+0000 7f3408400640 1 -- start start 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 0x7f34000776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f340010b350 0x7f3400078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3400114e50 con 0x7f340010a9d0 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3400114cd0 con 0x7f3400104fc0 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3408400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3400114fd0 con 0x7f340010b350 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44610/0 (socket says 172.21.15.67:44610) 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 -- 172.21.15.67:0/1397801388 learned_addr learned my addr 172.21.15.67:0/1397801388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405a00640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 0x7f34000776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 -- 172.21.15.67:0/1397801388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f340010b350 msgr2=0x7f3400078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f340010b350 0x7f3400078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 -- 172.21.15.67:0/1397801388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 msgr2=0x7f34000776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.706+0000 7f3405000640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 0x7f34000776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f3405000640 1 -- 172.21.15.67:0/1397801388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34001ab950 con 0x7f340010a9d0 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f3405000640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f33ec02a920 tx=0x7f33ec003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec03f070 con 0x7f340010a9d0 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34001abbe0 con 0x7f340010a9d0 2024-09-10T22:59:37.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33ec003e40 con 0x7f340010a9d0 2024-09-10T22:59:37.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec035030 con 0x7f340010a9d0 2024-09-10T22:59:37.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.707+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34001ac110 con 0x7f340010a9d0 2024-09-10T22:59:37.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.708+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34001053a0 con 0x7f340010a9d0 2024-09-10T22:59:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.712+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f33ec031b90 con 0x7f340010a9d0 2024-09-10T22:59:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.715+0000 7f33fe000640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 0x7f33e80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.715+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f33ec107240 con 0x7f340010a9d0 2024-09-10T22:59:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.715+0000 7f3405a00640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 0x7f33e80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.716+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f33ec0cdaf0 con 0x7f340010a9d0 2024-09-10T22:59:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.716+0000 7f3405a00640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 0x7f33e80b7880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33f400abe0 tx=0x7f33f400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:37.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.915+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f340002e5c0 con 0x7f340010a9d0 2024-09-10T22:59:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:37 smithi067 ceph-mon[30939]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T22:59:37.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.917+0000 7f33fe000640 1 -- 172.21.15.67:0/1397801388 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25221 (secure 0 0 0) 0x7f33ec0cd3f0 con 0x7f340010a9d0 2024-09-10T22:59:37.919 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:37.919 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T22:58:34.175811+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:59:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.919+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 msgr2=0x7f33e80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.919+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 0x7f33e80b7880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33f400abe0 tx=0x7f33f400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.919+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 msgr2=0x7f3400077bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.919+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f33ec02a920 tx=0x7f33ec003c20 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 shutdown_connections 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f33e80b53c0 0x7f33e80b7880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f340010b350 0x7f3400078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f340010a9d0 0x7f3400077bf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 --2- 172.21.15.67:0/1397801388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3400104fc0 0x7f34000776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 >> 172.21.15.67:0/1397801388 conn(0x7f34001009f0 msgr2=0x7f3400102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 shutdown_connections 2024-09-10T22:59:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:37.920+0000 7f3408400640 1 -- 172.21.15.67:0/1397801388 wait complete. 2024-09-10T22:59:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:37 smithi145 ceph-mon[34397]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T22:59:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:37 smithi092 ceph-mon[34223]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T22:59:38.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T22:59:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:38 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1397801388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:39.091 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:39.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:38 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1397801388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:39.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:38 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1397801388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:59:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:39 smithi067 ceph-mon[30939]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-09-10T22:59:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:39 smithi145 ceph-mon[34397]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-09-10T22:59:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:39 smithi092 ceph-mon[34223]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-09-10T22:59:40.489 INFO:teuthology.orchestra.run.smithi067.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T22:59:40.489 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T22:59:40.489 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 6190 2024-09-10T22:59:40.490 DEBUG:tasks.quiescer:Yielding 2024-09-10T22:59:40.490 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T22:59:40.505 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:59:40.505 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T22:59:40.505 INFO:tasks.workunit:timeout=3h 2024-09-10T22:59:40.505 INFO:tasks.workunit:cleanup=True 2024-09-10T22:59:40.506 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:40.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-10 22:57:20.478107706 +0000 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-10 22:57:20.478107706 +0000 2024-09-10T22:59:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-10 22:59:23.388533251 +0000 2024-09-10T22:59:40.544 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-09-10T22:59:40.544 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T22:59:40.544 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T22:59:40.653 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:59:40.690 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T22:59:40.900 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 -- 172.21.15.67:0/1370964756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108f90 msgr2=0x7f0c0010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 --2- 172.21.15.67:0/1370964756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108f90 0x7f0c0010f790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009ca0 tx=0x7f0bf802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 -- 172.21.15.67:0/1370964756 shutdown_connections 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 --2- 172.21.15.67:0/1370964756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108f90 0x7f0c0010f790 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 --2- 172.21.15.67:0/1370964756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00108610 0x7f0c00108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 --2- 172.21.15.67:0/1370964756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00102c00 0x7f0c00102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 -- 172.21.15.67:0/1370964756 >> 172.21.15.67:0/1370964756 conn(0x7f0c000fe6d0 msgr2=0x7f0c00100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 -- 172.21.15.67:0/1370964756 shutdown_connections 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.474+0000 7f0c06400640 1 -- 172.21.15.67:0/1370964756 wait complete. 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.475+0000 7f0c06400640 1 Processor -- start 2024-09-10T22:59:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.475+0000 7f0c06400640 1 -- start start 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00102c00 0x7f0c000754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00108f90 0x7f0c00075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00112b20 con 0x7f0c00108610 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001129a0 con 0x7f0c00102c00 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c06400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00112ca0 con 0x7f0c00108f90 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47904/0 (socket says 172.21.15.67:47904) 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 -- 172.21.15.67:0/1378218894 learned_addr learned my addr 172.21.15.67:0/1378218894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c05a00640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00108f90 0x7f0c00075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0c05000640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00102c00 0x7f0c000754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 -- 172.21.15.67:0/1378218894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00108f90 msgr2=0x7f0c00075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00108f90 0x7f0c00075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 -- 172.21.15.67:0/1378218894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00102c00 msgr2=0x7f0c000754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00102c00 0x7f0c000754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.476+0000 7f0bffe00640 1 -- 172.21.15.67:0/1378218894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001a73f0 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0bffe00640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00e9f0 tx=0x7f0bec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00cd80 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bec0108c0 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001a76e0 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0187c0 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.477+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a7c70 con 0x7f0c00108610 2024-09-10T22:59:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.478+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00102fe0 con 0x7f0c00108610 2024-09-10T22:59:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.482+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f0bec010430 con 0x7f0c00108610 2024-09-10T22:59:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.484+0000 7f0bfd600640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 0x7f0be40b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.484+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f0bec0e6b20 con 0x7f0c00108610 2024-09-10T22:59:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.484+0000 7f0c05000640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 0x7f0be40b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.485+0000 7f0c05000640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 0x7f0be40b79a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf000b590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.485+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0bec0ad370 con 0x7f0c00108610 2024-09-10T22:59:41.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.720+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c000767a0 con 0x7f0c00108610 2024-09-10T22:59:41.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.721+0000 7f0bfd600640 1 -- 172.21.15.67:0/1378218894 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11903 (secure 0 0 0) 0x7f0bec0acc70 con 0x7f0c00108610 2024-09-10T22:59:41.723 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T22:59:41.723 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":27,"btime":"2024-09-10T22:59:24:334283+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:59:23.805517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T22:59:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 27 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 msgr2=0x7f0be40b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 0x7f0be40b79a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf000b590 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 msgr2=0x7f0c000759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00e9f0 tx=0x7f0bec00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 shutdown_connections 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f0be40b54e0 0x7f0be40b79a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c00108f90 0x7f0c00075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c00108610 0x7f0c000759e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 --2- 172.21.15.67:0/1378218894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c00102c00 0x7f0c000754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 >> 172.21.15.67:0/1378218894 conn(0x7f0c000fe6d0 msgr2=0x7f0c000fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 shutdown_connections 2024-09-10T22:59:41.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:41.724+0000 7f0c06400640 1 -- 172.21.15.67:0/1378218894 wait complete. 2024-09-10T22:59:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:41 smithi067 ceph-mon[30939]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-10T22:59:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:41 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1378218894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:41 smithi145 ceph-mon[34397]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-10T22:59:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:41 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1378218894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:41 smithi092 ceph-mon[34223]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-09-10T22:59:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:41 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1378218894' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:56:19.886591+0000', 'modified': '2024-09-10T22:59:23.805517+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34730, 'mds_1': 34691, 'mds_2': 34631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24664': {'gid': 24664, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/2757139462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 2757139462}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 2757139462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/1285378679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 1285378679}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 1285378679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24691': {'gid': 24691, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6839/4174586003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 4174586003}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 4174586003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34631': {'gid': 34631, 'name': 'k', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.92:6833/307002386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 307002386}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 307002386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34691': {'gid': 34691, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.92:6837/523163273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 523163273}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 523163273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34730': {'gid': 34730, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/1949986898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1949986898}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1949986898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34730, 'qdb_cluster': [34631, 34691, 34730]} 2024-09-10T22:59:42.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T22:59:42.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:59:42.937 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:43.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 -- 172.21.15.67:0/2231112938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d410cf70 msgr2=0x7ff1d410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/2231112938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d410cf70 0x7ff1d410f3d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009ca0 tx=0x7ff1cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 -- 172.21.15.67:0/2231112938 shutdown_connections 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/2231112938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d410cf70 0x7ff1d410f3d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/2231112938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 0x7ff1d410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/2231112938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d4103b10 0x7ff1d4103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 -- 172.21.15.67:0/2231112938 >> 172.21.15.67:0/2231112938 conn(0x7ff1d40fd5b0 msgr2=0x7ff1d40ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 -- 172.21.15.67:0/2231112938 shutdown_connections 2024-09-10T22:59:43.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.500+0000 7ff1d9e00640 1 -- 172.21.15.67:0/2231112938 wait complete. 2024-09-10T22:59:43.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.501+0000 7ff1d9e00640 1 Processor -- start 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.501+0000 7ff1d9e00640 1 -- start start 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 0x7ff1d419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d410cf70 0x7ff1d41a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4112b20 con 0x7ff1d4103b10 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41129a0 con 0x7ff1d4104430 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4112ca0 con 0x7ff1d410cf70 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47934/0 (socket says 172.21.15.67:47934) 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d8a00640 1 -- 172.21.15.67:0/3443356308 learned_addr learned my addr 172.21.15.67:0/3443356308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1d9400640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d410cf70 0x7ff1d41a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.502+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 0x7ff1d419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 -- 172.21.15.67:0/3443356308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d410cf70 msgr2=0x7ff1d41a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d410cf70 0x7ff1d41a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 -- 172.21.15.67:0/3443356308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 msgr2=0x7ff1d419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 0x7ff1d419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 -- 172.21.15.67:0/3443356308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d41a1260 con 0x7ff1d4103b10 2024-09-10T22:59:43.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400b4a0 tx=0x7ff1c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d2000640 1 -- 172.21.15.67:0/3443356308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4014070 con 0x7ff1d4103b10 2024-09-10T22:59:43.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d41a1550 con 0x7ff1d4103b10 2024-09-10T22:59:43.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d2000640 1 -- 172.21.15.67:0/3443356308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1c4010aa0 con 0x7ff1d4103b10 2024-09-10T22:59:43.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d2000640 1 -- 172.21.15.67:0/3443356308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4017b70 con 0x7ff1d4103b10 2024-09-10T22:59:43.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.503+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41a7c10 con 0x7ff1d4103b10 2024-09-10T22:59:43.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.507+0000 7ff1ddfc1740 1 Processor -- start 2024-09-10T22:59:43.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 -- start start 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a7217c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559a720e7d50 con 0x559a72170d20 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a720d7180 con 0x559a72170400 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a720daf90 con 0x559a7217ccb0 2024-09-10T22:59:43.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.508+0000 7ff1d2000640 1 -- 172.21.15.67:0/3443356308 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7ff1c4010610 con 0x7ff1d4103b10 2024-09-10T22:59:43.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:56668/0 (socket says 172.21.15.67:56668) 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 -- 172.21.15.67:0/4125289673 learned_addr learned my addr 172.21.15.67:0/4125289673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a7217c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 -- 172.21.15.67:0/4125289673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 msgr2=0x559a7217f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 -- 172.21.15.67:0/4125289673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 msgr2=0x559a7217c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a7217c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d9400640 1 -- 172.21.15.67:0/4125289673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a720a1a70 con 0x559a72170400 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.509+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a7217c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:43.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.510+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009430 tx=0x7ff1cc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b07560a516822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.510+0000 7ff1c9c00640 1 -- 172.21.15.67:0/4125289673 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc031980 con 0x559a72170400 2024-09-10T22:59:43.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.510+0000 7ff1c9c00640 1 -- 172.21.15.67:0/4125289673 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1cc036040 con 0x559a72170400 2024-09-10T22:59:43.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.510+0000 7ff1c9c00640 1 -- 172.21.15.67:0/4125289673 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0044a0 con 0x559a72170400 2024-09-10T22:59:43.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1d2000640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 0x7ff1b80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4125289673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 msgr2=0x559a721707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009430 tx=0x7ff1cc009a20 comp rx=0 tx=0).stop 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4125289673 shutdown_connections 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a7217c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4125289673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a721707e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4125289673 >> 172.21.15.67:0/4125289673 conn(0x559a7216e230 msgr2=0x559a7216f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1d2000640 1 -- 172.21.15.67:0/3443356308 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff1c40e5c50 con 0x7ff1d4103b10 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 0x7ff1b80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4125289673 shutdown_connections 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.511+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4125289673 wait complete. 2024-09-10T22:59:43.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 -- start start 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1d8a00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 0x7ff1b80b7920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40abd30 tx=0x7ff1c40047d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 Processor -- start 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 0x559a7217a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559a71f385f0 con 0x559a72170400 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a72072460 con 0x559a72170d20 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a7217f9a0 con 0x559a7217ccb0 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.512+0000 7ff1ddfc1740 10 client.0 _ll_register_callbacks cb 0x559a72082210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47954/0 (socket says 172.21.15.67:47954) 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 -- 172.21.15.67:0/532988902 learned_addr learned my addr 172.21.15.67:0/532988902 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:43.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 0x559a7217a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 -- 172.21.15.67:0/532988902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 msgr2=0x559a7217a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 -- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 msgr2=0x559a7217a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 0x559a7217a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1d9400640 1 -- 172.21.15.67:0/532988902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a71e35540 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.513+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 0x559a7217a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1d9400640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003fa0 tx=0x7ff1cc003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af97994bd546f546 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1c2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc031d90 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff1cc004280 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a72074a70 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc003ce0 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1ddfc1740 10 client.15606 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a7208f6f0 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.514+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559a72074a70 con 0x559a72170400 2024-09-10T22:59:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1cc030cc0 con 0x559a72170400 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 10 client.15606 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 20 client.15606 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a7218d530 0x559a7218e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 4 client.15606 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a720e9ca0 con 0x559a7218d530 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.515+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a7218d530 0x559a7218e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.516+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a7218d530 0x559a7218e9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb3bcc8db8fbac2 server_cookie=81d7aa94e4a39d27 in_seq=0 out_seq=0 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.516+0000 7ff1c2000640 10 client.15606 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1c2000640 1 -- 172.21.15.67:0/532988902 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a720e9ca0 con 0x559a7218d530 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1c2000640 10 client.15606 handle_command_reply: tid=0 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 client.15606 shutdown 2024-09-10T22:59:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 msgr2=0x559a72179c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003fa0 tx=0x7ff1cc003f30 comp rx=0 tx=0).stop 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 shutdown_connections 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a7218d530 0x559a7218e9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a7217ccb0 0x559a7217a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170d20 0x559a7217a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/532988902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170400 0x559a72179c10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 >> 172.21.15.67:0/532988902 conn(0x559a7216e230 msgr2=0x559a72173dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1c2000640 0 client.15606 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.517+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 shutdown_connections 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.518+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/532988902 wait complete. 2024-09-10T22:59:43.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.518+0000 7ff1ddfc1740 20 client.15606 trim_cache size 0 max 16384 2024-09-10T22:59:43.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 Processor -- start 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 -- start start 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a7218dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170d20 0x559a72177050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559a7218cce0 con 0x559a7217ccb0 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a7218d3b0 con 0x559a72170400 2024-09-10T22:59:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a7218da80 con 0x559a72170d20 2024-09-10T22:59:43.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47960/0 (socket says 172.21.15.67:47960) 2024-09-10T22:59:43.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1d9400640 1 -- 172.21.15.67:0/3244582254 learned_addr learned my addr 172.21.15.67:0/3244582254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:43.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.537+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170d20 0x559a72177050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a7218dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 -- 172.21.15.67:0/3244582254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170d20 msgr2=0x559a72177050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170d20 0x559a72177050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 -- 172.21.15.67:0/3244582254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 msgr2=0x559a7218dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a7218dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 -- 172.21.15.67:0/3244582254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7216ecb0 con 0x559a7217ccb0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1d9400640 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003f00 tx=0x7ff1cc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8891fb4badcc9df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1c9200640 1 -- 172.21.15.67:0/3244582254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc009b20 con 0x559a7217ccb0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1c9200640 1 -- 172.21.15.67:0/3244582254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1cc04b040 con 0x559a7217ccb0 2024-09-10T22:59:43.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.538+0000 7ff1c9200640 1 -- 172.21.15.67:0/3244582254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc030720 con 0x559a7217ccb0 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/3244582254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 msgr2=0x559a72177590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc003f00 tx=0x7ff1cc009a20 comp rx=0 tx=0).stop 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/3244582254 shutdown_connections 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a7217ccb0 0x559a72177590 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170d20 0x559a72177050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/3244582254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a72170400 0x559a7218dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/3244582254 >> 172.21.15.67:0/3244582254 conn(0x559a7216e230 msgr2=0x559a7216f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/3244582254 shutdown_connections 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.539+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/3244582254 wait complete. 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 -- start start 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 Processor -- start 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170400 0x559a721972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 0x559a721789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559a7218e820 con 0x559a72170d20 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a7218e150 con 0x559a7217ccb0 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a7218eef0 con 0x559a72170400 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1ddfc1740 10 client.0 _ll_register_callbacks cb 0x7ff1cc003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:43.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.540+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47972/0 (socket says 172.21.15.67:47972) 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 -- 172.21.15.67:0/4006998154 learned_addr learned my addr 172.21.15.67:0/4006998154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170400 0x559a721972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 0x559a721789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 -- 172.21.15.67:0/4006998154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170400 msgr2=0x559a721972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170400 0x559a721972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 -- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 msgr2=0x559a721789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 0x559a721789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 -- 172.21.15.67:0/4006998154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a721698c0 con 0x559a72170d20 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 0x559a721789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fa0 tx=0x7ff1bc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3beae59d05667724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.541+0000 7ff1c2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc005020 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc04b040 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7ff1bc020040 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc031050 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a72195bf0 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1cc04b040 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1bc01e600 con 0x559a72170d20 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 10 client.15612 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 20 client.15612 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a721fe810 0x559a721fd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 4 client.15612 mds_command: new command op to 34631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:43.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.542+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 --> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] -- command(tid 0: {"prefix": "status"}) -- 0x559a720e5750 con 0x559a721fe810 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.543+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a721fe810 0x559a721fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1d9400640 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a721fe810 0x559a721fd520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53a8b04b80faf4bc server_cookie=b31cc763805c93e in_seq=0 out_seq=0 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1c2000640 10 client.15612 ms_handle_connect on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1c2000640 1 -- 172.21.15.67:0/4006998154 <== mds.2 v2:172.21.15.92:6832/307002386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559a720e5750 con 0x559a721fe810 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1c2000640 10 client.15612 handle_command_reply: tid=0 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 client.15612 shutdown 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 msgr2=0x559a72178490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fa0 tx=0x7ff1bc012820 comp rx=0 tx=0).stop 2024-09-10T22:59:43.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 shutdown_connections 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:6832/307002386,v1:172.21.15.92:6833/307002386] conn(0x559a721fe810 0x559a721fd520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a7217ccb0 0x559a721789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559a72170d20 0x559a72178490 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 --2- 172.21.15.67:0/4006998154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a72170400 0x559a721972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 >> 172.21.15.67:0/4006998154 conn(0x559a7216e230 msgr2=0x559a72177050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1c2000640 0 client.15612 ms_handle_reset on v2:172.21.15.92:6832/307002386 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 shutdown_connections 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.544+0000 7ff1ddfc1740 1 -- 172.21.15.67:0/4006998154 wait complete. 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1ddfc1740 20 client.15612 trim_cache size 0 max 16384 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:43.550 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34631, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 153.132244947, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 27, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 252.535508505 2024-09-10T22:59:43.551 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 msgr2=0x7ff1b80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 0x7ff1b80b7920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40abd30 tx=0x7ff1c40047d0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 msgr2=0x7ff1d419b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400b4a0 tx=0x7ff1c400b960 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 shutdown_connections 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff1b80b5460 0x7ff1b80b7920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1d410cf70 0x7ff1d41a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1d4104430 0x7ff1d419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 --2- 172.21.15.67:0/3443356308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d4103b10 0x7ff1d419b1b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.545+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 >> 172.21.15.67:0/3443356308 conn(0x7ff1d40fd5b0 msgr2=0x7ff1d40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.546+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 shutdown_connections 2024-09-10T22:59:43.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:43.546+0000 7ff1d9e00640 1 -- 172.21.15.67:0/3443356308 wait complete. 2024-09-10T22:59:44.074 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:43 smithi092 ceph-mon[34223]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T22:59:44.114 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:43 smithi067 ceph-mon[30939]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T22:59:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:43 smithi145 ceph-mon[34397]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T22:59:44.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:59:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 -- 172.21.15.67:0/4060084951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075540 msgr2=0x7f963c075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 --2- 172.21.15.67:0/4060084951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075540 0x7f963c075920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9630009900 tx=0x7f963002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 -- 172.21.15.67:0/4060084951 shutdown_connections 2024-09-10T22:59:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 --2- 172.21.15.67:0/4060084951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 --2- 172.21.15.67:0/4060084951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075e60 0x7f963c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 --2- 172.21.15.67:0/4060084951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075540 0x7f963c075920 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 -- 172.21.15.67:0/4060084951 >> 172.21.15.67:0/4060084951 conn(0x7f963c0fe830 msgr2=0x7f963c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 -- 172.21.15.67:0/4060084951 shutdown_connections 2024-09-10T22:59:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.325+0000 7f9643600640 1 -- 172.21.15.67:0/4060084951 wait complete. 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 Processor -- start 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 -- start start 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075540 0x7f963c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1192c0 con 0x7f963c075e60 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f963c119140 con 0x7f963c113b20 2024-09-10T22:59:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.326+0000 7f9643600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f963c119440 con 0x7f963c075540 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f9640c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075540 0x7f963c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47986/0 (socket says 172.21.15.67:47986) 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 -- 172.21.15.67:0/441266792 learned_addr learned my addr 172.21.15.67:0/441266792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f9641600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 -- 172.21.15.67:0/441266792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075540 msgr2=0x7f963c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075540 0x7f963c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 -- 172.21.15.67:0/441266792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 msgr2=0x7f963c1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f963be00640 1 -- 172.21.15.67:0/441266792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a5900 con 0x7f963c075e60 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.327+0000 7f9641600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f963be00640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f962800d770 tx=0x7f962800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f9639600640 1 -- 172.21.15.67:0/441266792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628017070 con 0x7f963c075e60 2024-09-10T22:59:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f9639600640 1 -- 172.21.15.67:0/441266792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96280054b0 con 0x7f963c075e60 2024-09-10T22:59:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a5bf0 con 0x7f963c075e60 2024-09-10T22:59:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f9639600640 1 -- 172.21.15.67:0/441266792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628004c60 con 0x7f963c075e60 2024-09-10T22:59:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.328+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1ac2b0 con 0x7f963c075e60 2024-09-10T22:59:45.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.331+0000 7f964621e740 1 Processor -- start 2024-09-10T22:59:45.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.331+0000 7f964621e740 1 -- start start 2024-09-10T22:59:45.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.331+0000 7f9639600640 1 -- 172.21.15.67:0/441266792 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f9628004e00 con 0x7f963c075e60 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 0x556838577620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x5568385835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 0x556838585ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5568384eed50 con 0x556838583af0 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5568384de180 con 0x556838577b60 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f964621e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5568384e1f90 con 0x556838577240 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f963be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 0x556838585ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f9641600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 0x556838577620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f963be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 0x556838585ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47988/0 (socket says 172.21.15.67:47988) 2024-09-10T22:59:45.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f963be00640 1 -- 172.21.15.67:0/4160134875 learned_addr learned my addr 172.21.15.67:0/4160134875 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:45.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.332+0000 7f9640c00640 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x5568385835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 -- 172.21.15.67:0/4160134875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 msgr2=0x5568385835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x5568385835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 -- 172.21.15.67:0/4160134875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 msgr2=0x556838585ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 0x556838585ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 -- 172.21.15.67:0/4160134875 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568384a8a70 con 0x556838577240 2024-09-10T22:59:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f9641600640 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 0x556838577620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9634009de0 tx=0x7f963400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d017c0764f1797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f962dc00640 1 -- 172.21.15.67:0/4160134875 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634004b20 con 0x556838577240 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f962dc00640 1 -- 172.21.15.67:0/4160134875 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9634004cc0 con 0x556838577240 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.333+0000 7f962dc00640 1 -- 172.21.15.67:0/4160134875 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340117c0 con 0x556838577240 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 -- 172.21.15.67:0/4160134875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 msgr2=0x556838577620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 0x556838577620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9634009de0 tx=0x7f963400f730 comp rx=0 tx=0).stop 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 -- 172.21.15.67:0/4160134875 shutdown_connections 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838583af0 0x556838585ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x5568385835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 --2- 172.21.15.67:0/4160134875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577240 0x556838577620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 -- 172.21.15.67:0/4160134875 >> 172.21.15.67:0/4160134875 conn(0x556838575020 msgr2=0x556838576210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f9639600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 0x7f96240b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 -- 172.21.15.67:0/4160134875 shutdown_connections 2024-09-10T22:59:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f9639600640 1 -- 172.21.15.67:0/441266792 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f96280e5d40 con 0x7f963c075e60 2024-09-10T22:59:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.334+0000 7f964621e740 1 -- 172.21.15.67:0/4160134875 wait complete. 2024-09-10T22:59:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.335+0000 7f9640c00640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 0x7f96240b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.335+0000 7f964621e740 1 -- start start 2024-09-10T22:59:45.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.335+0000 7f964621e740 1 Processor -- start 2024-09-10T22:59:45.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.335+0000 7f9640c00640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 0x7f96240b7880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9630006010 tx=0x7f9630005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577b60 0x556838581000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838583af0 0x556838581540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55683833f5f0 con 0x556838577240 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556838479460 con 0x556838583af0 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556838586840 con 0x556838577b60 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f964621e740 10 client.0 _ll_register_callbacks cb 0x556838489210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47992/0 (socket says 172.21.15.67:47992) 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f9640c00640 1 -- 172.21.15.67:0/2616745942 learned_addr learned my addr 172.21.15.67:0/2616745942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f9641600640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838583af0 0x556838581540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f963be00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577b60 0x556838581000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.336+0000 7f9640c00640 1 -- 172.21.15.67:0/2616745942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577b60 msgr2=0x556838581000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9640c00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577b60 0x556838581000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9640c00640 1 -- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838583af0 msgr2=0x556838581540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9640c00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838583af0 0x556838581540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9640c00640 1 -- 172.21.15.67:0/2616745942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683823c540 con 0x556838577240 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9640c00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9630033000 tx=0x7f9630005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aa6da8b964fbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9622000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963004e070 con 0x556838577240 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f963004a860 con 0x556838577240 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.337+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55683847ba70 con 0x556838577240 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f963005a050 con 0x556838577240 2024-09-10T22:59:45.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 10 client.15624 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568384966f0 con 0x556838577240 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55683847ba70 con 0x556838577240 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f963003d9d0 con 0x556838577240 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 10 client.15624 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 20 client.15624 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x556838594870 0x556838595ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 4 client.15624 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:45.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.338+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568384f0ca0 con 0x556838594870 2024-09-10T22:59:45.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.339+0000 7f963be00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x556838594870 0x556838595ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.340+0000 7f963be00640 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x556838594870 0x556838595ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6763757147da77f5 server_cookie=255454ecf1d783c in_seq=0 out_seq=0 2024-09-10T22:59:45.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.340+0000 7f9622000640 10 client.15624 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f9622000640 1 -- 172.21.15.67:0/2616745942 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568384f0ca0 con 0x556838594870 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f9622000640 10 client.15624 handle_command_reply: tid=0 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 client.15624 shutdown 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 msgr2=0x556838580ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9630033000 tx=0x7f9630005c00 comp rx=0 tx=0).stop 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 shutdown_connections 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x556838594870 0x556838595ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838583af0 0x556838581540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838577b60 0x556838581000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 --2- 172.21.15.67:0/2616745942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838580ac0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 >> 172.21.15.67:0/2616745942 conn(0x556838575020 msgr2=0x55683857abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f9622000640 0 client.15624 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:45.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.341+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 shutdown_connections 2024-09-10T22:59:45.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.342+0000 7f964621e740 1 -- 172.21.15.67:0/2616745942 wait complete. 2024-09-10T22:59:45.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.342+0000 7f964621e740 20 client.15624 trim_cache size 0 max 16384 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.360+0000 7f964621e740 1 Processor -- start 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 -- start start 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x55683857b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x55683857bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556838594020 con 0x556838577240 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5568385946f0 con 0x556838577b60 2024-09-10T22:59:45.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f964621e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556838594dc0 con 0x556838583af0 2024-09-10T22:59:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48002/0 (socket says 172.21.15.67:48002) 2024-09-10T22:59:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f9640c00640 1 -- 172.21.15.67:0/731518327 learned_addr learned my addr 172.21.15.67:0/731518327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f9641600640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x55683857bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.361+0000 7f963be00640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x55683857b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 -- 172.21.15.67:0/731518327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 msgr2=0x55683857bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x55683857bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 -- 172.21.15.67:0/731518327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 msgr2=0x55683857b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x55683857b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 -- 172.21.15.67:0/731518327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568385802e0 con 0x556838577240 2024-09-10T22:59:45.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f963be00640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x55683857b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f9640c00640 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f963002a430 tx=0x7f9630003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6db92e7d7e1798 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f962d200640 1 -- 172.21.15.67:0/731518327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963004e040 con 0x556838577240 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f962d200640 1 -- 172.21.15.67:0/731518327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9630063070 con 0x556838577240 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.362+0000 7f962d200640 1 -- 172.21.15.67:0/731518327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630035040 con 0x556838577240 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- 172.21.15.67:0/731518327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 msgr2=0x556838594f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f963002a430 tx=0x7f9630003c40 comp rx=0 tx=0).stop 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- 172.21.15.67:0/731518327 shutdown_connections 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x55683857bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x55683857b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 --2- 172.21.15.67:0/731518327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x556838594f40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- 172.21.15.67:0/731518327 >> 172.21.15.67:0/731518327 conn(0x556838575020 msgr2=0x55683856fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- 172.21.15.67:0/731518327 shutdown_connections 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- 172.21.15.67:0/731518327 wait complete. 2024-09-10T22:59:45.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.363+0000 7f964621e740 1 -- start start 2024-09-10T22:59:45.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 Processor -- start 2024-09-10T22:59:45.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x556838597c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x5568385981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556838595b60 con 0x556838577240 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556838595490 con 0x556838577b60 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556838596230 con 0x556838583af0 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f964621e740 10 client.0 _ll_register_callbacks cb 0x7f9630004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f9640c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48008/0 (socket says 172.21.15.67:48008) 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f9640c00640 1 -- 172.21.15.67:0/3502062092 learned_addr learned my addr 172.21.15.67:0/3502062092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.364+0000 7f963be00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x556838597c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9641600640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x5568385981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 -- 172.21.15.67:0/3502062092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 msgr2=0x5568385981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x5568385981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 -- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 msgr2=0x556838597c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x556838597c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 -- 172.21.15.67:0/3502062092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568385989e0 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9640c00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f963002a910 tx=0x7f963003d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206ba263f3367fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9622000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963006c060 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.365+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9630063070 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f963004a660 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f963004e070 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 10 client.15630 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556838601270 con 0x556838577240 2024-09-10T22:59:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9630063070 con 0x556838577240 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f96300538d0 con 0x556838577240 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 10 client.15630 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 20 client.15630 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5568386059d0 0x5568386046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 4 client.15630 mds_command: new command op to 34691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.366+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 --> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] -- command(tid 0: {"prefix": "status"}) -- 0x5568384ec750 con 0x5568386059d0 2024-09-10T22:59:45.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.367+0000 7f963be00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5568386059d0 0x5568386046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.367+0000 7f963be00640 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5568386059d0 0x5568386046e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f750eafde5a7b73e server_cookie=b21520fc49ff8259 in_seq=0 out_seq=0 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.367+0000 7f9622000640 10 client.15630 ms_handle_connect on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f9622000640 1 -- 172.21.15.67:0/3502062092 <== mds.1 v2:172.21.15.92:6836/523163273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5568384ec750 con 0x5568386059d0 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f9622000640 10 client.15630 handle_command_reply: tid=0 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 client.15630 shutdown 2024-09-10T22:59:45.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 msgr2=0x55683859e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f963002a910 tx=0x7f963003d700 comp rx=0 tx=0).stop 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 shutdown_connections 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:6836/523163273,v1:172.21.15.92:6837/523163273] conn(0x5568386059d0 0x5568386046e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556838583af0 0x5568385981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556838577b60 0x556838597c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 --2- 172.21.15.67:0/3502062092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556838577240 0x55683859e760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 >> 172.21.15.67:0/3502062092 conn(0x556838575020 msgr2=0x55683857bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f9622000640 0 client.15630 ms_handle_reset on v2:172.21.15.92:6836/523163273 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 shutdown_connections 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 1 -- 172.21.15.67:0/3502062092 wait complete. 2024-09-10T22:59:45.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.368+0000 7f964621e740 20 client.15630 trim_cache size 0 max 16384 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34691, 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 155.96815485299999, 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 27, 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:45.374 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 233.16264090000001 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 msgr2=0x7f96240b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 0x7f96240b7880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9630006010 tx=0x7f9630005e80 comp rx=0 tx=0).stop 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 msgr2=0x7f963c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f962800d770 tx=0x7f962800dc30 comp rx=0 tx=0).stop 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 shutdown_connections 2024-09-10T22:59:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f96240b53c0 0x7f96240b7880 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f963c113b20 0x7f963c1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963c075e60 0x7f963c19fd90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 --2- 172.21.15.67:0/441266792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f963c075540 0x7f963c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 >> 172.21.15.67:0/441266792 conn(0x7f963c0fe830 msgr2=0x7f963c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 shutdown_connections 2024-09-10T22:59:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:45.369+0000 7f9643600640 1 -- 172.21.15.67:0/441266792 wait complete. 2024-09-10T22:59:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:45 smithi145 ceph-mon[34397]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T22:59:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:45 smithi092 ceph-mon[34223]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T22:59:46.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:45 smithi067 ceph-mon[30939]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T22:59:46.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:59:46.769 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:47.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 -- 172.21.15.67:0/1209609239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d3810a820 msgr2=0x7f3d3810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/1209609239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d3810a820 0x7f3d3810ac60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009900 tx=0x7f3d2402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 -- 172.21.15.67:0/1209609239 shutdown_connections 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/1209609239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810b1a0 0x7f3d381119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/1209609239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d3810a820 0x7f3d3810ac60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/1209609239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38104e10 0x7f3d381051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.372+0000 7f3d3ce00640 1 -- 172.21.15.67:0/1209609239 >> 172.21.15.67:0/1209609239 conn(0x7f3d38100840 msgr2=0x7f3d38102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.373+0000 7f3d3ce00640 1 -- 172.21.15.67:0/1209609239 shutdown_connections 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.373+0000 7f3d3ce00640 1 -- 172.21.15.67:0/1209609239 wait complete. 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.374+0000 7f3d3ce00640 1 Processor -- start 2024-09-10T22:59:47.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.374+0000 7f3d3ce00640 1 -- start start 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810a820 0x7f3d3819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 0x7f3d381a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38114cd0 con 0x7f3d38104e10 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38114b50 con 0x7f3d3810a820 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d3ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38114e50 con 0x7f3d3810b1a0 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48022/0 (socket says 172.21.15.67:48022) 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 -- 172.21.15.67:0/496180437 learned_addr learned my addr 172.21.15.67:0/496180437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d36a00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810a820 0x7f3d3819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37e00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 0x7f3d381a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 -- 172.21.15.67:0/496180437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 msgr2=0x7f3d381a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 0x7f3d381a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 -- 172.21.15.67:0/496180437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810a820 msgr2=0x7f3d3819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810a820 0x7f3d3819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37400640 1 -- 172.21.15.67:0/496180437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381a11b0 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.375+0000 7f3d37e00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 0x7f3d381a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d37400640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d2fe00640 1 -- 172.21.15.67:0/496180437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800cd80 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d2fe00640 1 -- 172.21.15.67:0/496180437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d280054b0 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d381a14a0 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d2fe00640 1 -- 172.21.15.67:0/496180437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28010430 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.376+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d381a7b60 con 0x7f3d38104e10 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.379+0000 7f3d4103f740 1 Processor -- start 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 -- start start 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d27472b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d27472e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d274696d50 con 0x55d27471f400 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d274686180 con 0x55d27472bcb0 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d4103f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d274689f90 con 0x55d27471fd20 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48028/0 (socket says 172.21.15.67:48028) 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.380+0000 7f3d37e00640 1 -- 172.21.15.67:0/3751694026 learned_addr learned my addr 172.21.15.67:0/3751694026 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d36a00640 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d27472e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 -- 172.21.15.67:0/3751694026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 msgr2=0x55d27472b770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d27472b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 -- 172.21.15.67:0/3751694026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 msgr2=0x55d27472e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d27472e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 -- 172.21.15.67:0/3751694026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d274650a70 con 0x55d27471f400 2024-09-10T22:59:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d2fe00640 1 -- 172.21.15.67:0/496180437 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f3d28005000 con 0x7f3d38104e10 2024-09-10T22:59:47.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.381+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009fc0 tx=0x7f3d3000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8746ae8845c2b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.382+0000 7f3d2cc00640 1 -- 172.21.15.67:0/3751694026 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30003b70 con 0x55d27471f400 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.382+0000 7f3d2cc00640 1 -- 172.21.15.67:0/3751694026 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d30003d10 con 0x55d27471f400 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.382+0000 7f3d2cc00640 1 -- 172.21.15.67:0/3751694026 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300118e0 con 0x55d27471f400 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d2fe00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 0x7f3d1c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 -- 172.21.15.67:0/3751694026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 msgr2=0x55d27471f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009fc0 tx=0x7f3d3000f7f0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d2fe00640 1 -- 172.21.15.67:0/496180437 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f3d280e65d0 con 0x7f3d38104e10 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d37400640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 0x7f3d1c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 -- 172.21.15.67:0/3751694026 shutdown_connections 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d27472e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d27472b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3751694026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d27471f7e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 -- 172.21.15.67:0/3751694026 >> 172.21.15.67:0/3751694026 conn(0x55d27471d230 msgr2=0x55d27471e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.383+0000 7f3d4103f740 1 -- 172.21.15.67:0/3751694026 shutdown_connections 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.384+0000 7f3d4103f740 1 -- 172.21.15.67:0/3751694026 wait complete. 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.384+0000 7f3d37400640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 0x7f3d1c0b79a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55d274722580 tx=0x7f3d280056f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.384+0000 7f3d4103f740 1 -- start start 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 Processor -- start 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274728d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d2747292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d2747297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2744e75f0 con 0x55d27471f400 2024-09-10T22:59:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d274621460 con 0x55d27472bcb0 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d27472ea00 con 0x55d27471fd20 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.385+0000 7f3d4103f740 10 client.0 _ll_register_callbacks cb 0x55d274631210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274728d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274728d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48044/0 (socket says 172.21.15.67:48044) 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d37e00640 1 -- 172.21.15.67:0/2155373750 learned_addr learned my addr 172.21.15.67:0/2155373750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d2747292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d37e00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d2747297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 -- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 msgr2=0x55d2747297e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d2747297e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 -- 172.21.15.67:0/2155373750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 msgr2=0x55d274728d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274728d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d36a00640 1 -- 172.21.15.67:0/2155373750 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2743e4540 con 0x55d27471fd20 2024-09-10T22:59:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.386+0000 7f3d37e00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d2747297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.387+0000 7f3d36a00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d2747292a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005940 tx=0x7f3d24031dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9884041ba481ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.387+0000 7f3d21600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.387+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24035040 con 0x55d27471fd20 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.387+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f3d24031810 con 0x55d27471fd20 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.387+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d274623a70 con 0x55d27471fd20 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.388+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d2404f050 con 0x55d27471fd20 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.388+0000 7f3d4103f740 10 client.25372 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.388+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d27463e6f0 con 0x55d27471fd20 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.388+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d274623a70 con 0x55d27471fd20 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d24044020 con 0x55d27471fd20 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 10 client.25372 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 10 client.25372 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 20 client.25372 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747452b0 0x55d274746720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 4 client.25372 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d274698ca0 con 0x55d2747452b0 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.389+0000 7f3d37e00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747452b0 0x55d274746720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.390+0000 7f3d37e00640 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747452b0 0x55d274746720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4ddb4d8fbd1deb server_cookie=df8eb46625940eef in_seq=0 out_seq=0 2024-09-10T22:59:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.390+0000 7f3d21600640 10 client.25372 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d21600640 1 -- 172.21.15.67:0/2155373750 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d274698ca0 con 0x55d2747452b0 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d21600640 10 client.25372 handle_command_reply: tid=0 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 client.25372 shutdown 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 msgr2=0x55d2747292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d2747292a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005940 tx=0x7f3d24031dc0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 shutdown_connections 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747452b0 0x55d274746720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27472bcb0 0x55d2747297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27471fd20 0x55d2747292a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d21600640 0 client.25372 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 --2- 172.21.15.67:0/2155373750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274728d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.391+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 >> 172.21.15.67:0/2155373750 conn(0x55d27471d230 msgr2=0x55d274722dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.392+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 shutdown_connections 2024-09-10T22:59:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.392+0000 7f3d4103f740 1 -- 172.21.15.67:0/2155373750 wait complete. 2024-09-10T22:59:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.392+0000 7f3d4103f740 20 client.25372 trim_cache size 0 max 16384 2024-09-10T22:59:47.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 Processor -- start 2024-09-10T22:59:47.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 -- start start 2024-09-10T22:59:47.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 0x55d274723ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d274724530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d274744a60 con 0x55d27471f400 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d274745130 con 0x55d27471fd20 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d4103f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d274745800 con 0x55d27472bcb0 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.415+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48060/0 (socket says 172.21.15.67:48060) 2024-09-10T22:59:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 -- 172.21.15.67:0/3762827587 learned_addr learned my addr 172.21.15.67:0/3762827587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d36a00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 0x55d274723ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d274724530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 -- 172.21.15.67:0/3762827587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 msgr2=0x55d274724530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d274724530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 -- 172.21.15.67:0/3762827587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 msgr2=0x55d274723ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 0x55d274723ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d37e00640 1 -- 172.21.15.67:0/3762827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27471fb50 con 0x55d27471f400 2024-09-10T22:59:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.416+0000 7f3d36a00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 0x55d274723ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.417+0000 7f3d37e00640 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009f90 tx=0x7f3d30009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f6573971db7294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.417+0000 7f3d23e00640 1 -- 172.21.15.67:0/3762827587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30024040 con 0x55d27471f400 2024-09-10T22:59:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.417+0000 7f3d23e00640 1 -- 172.21.15.67:0/3762827587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d30011c00 con 0x55d27471f400 2024-09-10T22:59:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.417+0000 7f3d23e00640 1 -- 172.21.15.67:0/3762827587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3002c970 con 0x55d27471f400 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- 172.21.15.67:0/3762827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 msgr2=0x55d274746050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009f90 tx=0x7f3d30009500 comp rx=0 tx=0).stop 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- 172.21.15.67:0/3762827587 shutdown_connections 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d274724530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471fd20 0x55d274723ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 --2- 172.21.15.67:0/3762827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471f400 0x55d274746050 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- 172.21.15.67:0/3762827587 >> 172.21.15.67:0/3762827587 conn(0x55d27471d230 msgr2=0x55d27471e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- 172.21.15.67:0/3762827587 shutdown_connections 2024-09-10T22:59:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- 172.21.15.67:0/3762827587 wait complete. 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.418+0000 7f3d4103f740 1 -- start start 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 Processor -- start 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 0x55d274748b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d2747495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2747465a0 con 0x55d27471fd20 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d274745ed0 con 0x55d27471f400 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d274746c70 con 0x55d27472bcb0 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d4103f740 10 client.0 _ll_register_callbacks cb 0x7f3d24031a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.419+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 0x55d274748b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48068/0 (socket says 172.21.15.67:48068) 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 -- 172.21.15.67:0/1944363029 learned_addr learned my addr 172.21.15.67:0/1944363029 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d37e00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d2747495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 -- 172.21.15.67:0/1944363029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 msgr2=0x55d2747495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d2747495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 -- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 msgr2=0x55d274748b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 0x55d274748b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 -- 172.21.15.67:0/1944363029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27474e460 con 0x55d27471fd20 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d37e00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 0x55d274748b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d36a00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3d24003e40 tx=0x7f3d24005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209f7ad389f46066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.420+0000 7f3d21600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T22:59:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24058060 con 0x55d27471fd20 2024-09-10T22:59:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f3d240035f0 con 0x55d27471fd20 2024-09-10T22:59:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d30011c00 con 0x55d27471fd20 2024-09-10T22:59:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d2405f050 con 0x55d27471fd20 2024-09-10T22:59:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d4103f740 10 client.15648 fetch_fsmap learned FSMap version 27 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2747b2110 con 0x55d27471fd20 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.421+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d30011c00 con 0x55d27471fd20 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d24044040 con 0x55d27471fd20 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 10 client.15648 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 10 client.15648 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 20 client.15648 populate_metadata read hostname 'smithi067' 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747b6ce0 0x55d2747b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 4 client.15648 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.422+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "status"}) -- 0x55d274694750 con 0x55d2747b6ce0 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d37e00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747b6ce0 0x55d2747b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d37e00640 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747b6ce0 0x55d2747b59f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0873f0f7c79f29 server_cookie=d2f37c5393041382 in_seq=0 out_seq=0 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d21600640 10 client.15648 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:47.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d21600640 1 -- 172.21.15.67:0/1944363029 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d274694750 con 0x55d2747b6ce0 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d21600640 10 client.15648 handle_command_reply: tid=0 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.423+0000 7f3d4103f740 1 client.15648 shutdown 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 msgr2=0x55d274749060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3d24003e40 tx=0x7f3d24005fd0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 shutdown_connections 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d2747b6ce0 0x55d2747b59f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d27472bcb0 0x55d2747495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d27471fd20 0x55d274749060 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 --2- 172.21.15.67:0/1944363029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27471f400 0x55d274748b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 >> 172.21.15.67:0/1944363029 conn(0x55d27471d230 msgr2=0x55d27471e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d21600640 0 client.15648 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 shutdown_connections 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 1 -- 172.21.15.67:0/1944363029 wait complete. 2024-09-10T22:59:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.424+0000 7f3d4103f740 20 client.15648 trim_cache size 0 max 16384 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "5c4a3ee4-6fc6-11ef-bcea-c7b262605968", 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 34730, 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 207.52678865499999, 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 27, 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-09-10T22:59:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T22:59:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 219.879353353 2024-09-10T22:59:47.430 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T22:59:47.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 msgr2=0x7f3d1c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 0x7f3d1c0b79a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55d274722580 tx=0x7f3d280056f0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 msgr2=0x7f3d3819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 shutdown_connections 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d1c0b54e0 0x7f3d1c0b79a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d3810b1a0 0x7f3d381a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d3810a820 0x7f3d3819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 --2- 172.21.15.67:0/496180437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d38104e10 0x7f3d3819b100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.425+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 >> 172.21.15.67:0/496180437 conn(0x7f3d38100840 msgr2=0x7f3d38102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.426+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 shutdown_connections 2024-09-10T22:59:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T22:59:47.426+0000 7f3d3ce00640 1 -- 172.21.15.67:0/496180437 wait complete. 2024-09-10T22:59:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:47 smithi145 ceph-mon[34397]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T22:59:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:47 smithi092 ceph-mon[34223]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T22:59:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:47 smithi067 ceph-mon[30939]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T22:59:48.388 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T22:59:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:49 smithi145 ceph-mon[34397]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T22:59:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:49 smithi092 ceph-mon[34223]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T22:59:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:49 smithi067 ceph-mon[30939]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T22:59:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:51 smithi145 ceph-mon[34397]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T22:59:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:51 smithi092 ceph-mon[34223]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T22:59:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:51 smithi067 ceph-mon[30939]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T22:59:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:53 smithi145 ceph-mon[34397]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T22:59:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:53 smithi092 ceph-mon[34223]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T22:59:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:53 smithi067 ceph-mon[30939]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T22:59:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:55 smithi145 ceph-mon[34397]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:59:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:55 smithi092 ceph-mon[34223]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:59:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:55 smithi067 ceph-mon[30939]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T22:59:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:57 smithi145 ceph-mon[34397]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T22:59:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:57 smithi092 ceph-mon[34223]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T22:59:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:57 smithi067 ceph-mon[30939]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T23:00:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 22:59:59 smithi145 ceph-mon[34397]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T23:00:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 22:59:59 smithi092 ceph-mon[34223]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T23:00:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 22:59:59 smithi067 ceph-mon[30939]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T23:00:00.798 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:00:00.486+0000 7fc9ecc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:00:00.798 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:00:00.486+0000 7fc9ecc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:00:00.799 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:00:00.486+0000 7fc9ecc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:00:00.799 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:00:00.486+0000 7fc9ecc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:00:00.799 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:00:00.486+0000 7fc9ecc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:00:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:00 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T23:00:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:00 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T23:00:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:00 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T23:00:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:01 smithi145 ceph-mon[34397]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T23:00:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:01 smithi092 ceph-mon[34223]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T23:00:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:01 smithi067 ceph-mon[30939]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T23:00:03.388 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T23:00:03.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:00:03.807 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:03 smithi145 ceph-mon[34397]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-10T23:00:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:03 smithi145 ceph-mon[34397]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T23:00:04.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:03 smithi067 ceph-mon[30939]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-10T23:00:04.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:03 smithi067 ceph-mon[30939]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T23:00:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:03 smithi092 ceph-mon[34223]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-09-10T23:00:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:03 smithi092 ceph-mon[34223]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T23:00:04.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 -- 172.21.15.67:0/466996169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 msgr2=0x7fa65c111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:04.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 --2- 172.21.15.67:0/466996169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c111610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa654009ca0 tx=0x7fa65402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 -- 172.21.15.67:0/466996169 shutdown_connections 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 --2- 172.21.15.67:0/466996169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c111610 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 --2- 172.21.15.67:0/466996169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c075de0 0x7fa65c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 --2- 172.21.15.67:0/466996169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c0754c0 0x7fa65c0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 -- 172.21.15.67:0/466996169 >> 172.21.15.67:0/466996169 conn(0x7fa65c0fe660 msgr2=0x7fa65c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 -- 172.21.15.67:0/466996169 shutdown_connections 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.458+0000 7fa661200640 1 -- 172.21.15.67:0/466996169 wait complete. 2024-09-10T23:00:04.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.459+0000 7fa661200640 1 Processor -- start 2024-09-10T23:00:04.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.459+0000 7fa661200640 1 -- start start 2024-09-10T23:00:04.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:04.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c075de0 0x7fa65c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:04.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c114d00 con 0x7fa65c10f220 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c114b80 con 0x7fa65c075de0 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa661200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c114e80 con 0x7fa65c0754c0 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.460+0000 7fa65b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:49634/0 (socket says 172.21.15.67:49634) 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 -- 172.21.15.67:0/349216664 learned_addr learned my addr 172.21.15.67:0/349216664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65aa00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c075de0 0x7fa65c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65be00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 -- 172.21.15.67:0/349216664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c075de0 msgr2=0x7fa65c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c075de0 0x7fa65c19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 -- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 msgr2=0x7fa65c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65b400640 1 -- 172.21.15.67:0/349216664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c1a11e0 con 0x7fa65c0754c0 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.461+0000 7fa65be00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa65b400640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d950 tx=0x7fa64c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c017030 con 0x7fa65c0754c0 2024-09-10T23:00:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa64c004020 con 0x7fa65c0754c0 2024-09-10T23:00:04.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0055d0 con 0x7fa65c0754c0 2024-09-10T23:00:04.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c1a14d0 con 0x7fa65c0754c0 2024-09-10T23:00:04.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.462+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c1a7b90 con 0x7fa65c0754c0 2024-09-10T23:00:04.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.463+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c0758a0 con 0x7fa65c0754c0 2024-09-10T23:00:04.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.468+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa64c009a30 con 0x7fa65c0754c0 2024-09-10T23:00:04.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.469+0000 7fa653e00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 0x7fa6400b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:04.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.469+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa64c0e64f0 con 0x7fa65c0754c0 2024-09-10T23:00:04.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.469+0000 7fa65aa00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 0x7fa6400b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:04.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.470+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa64c0acd00 con 0x7fa65c0754c0 2024-09-10T23:00:04.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.470+0000 7fa65aa00640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 0x7fa6400b7a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa648005fd0 tx=0x7fa648005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.810+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65c1a7ff0 con 0x7fa65c0754c0 2024-09-10T23:00:04.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.812+0000 7fa653e00640 1 -- 172.21.15.67:0/349216664 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11903 (secure 0 0 0) 0x7fa64c0ac600 con 0x7fa65c0754c0 2024-09-10T23:00:04.814 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:00:04.814 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":27,"btime":"2024-09-10T22:59:24:334283+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:59:23.805517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:00:04.814 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 27 2024-09-10T23:00:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 msgr2=0x7fa6400b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 0x7fa6400b7a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa648005fd0 tx=0x7fa648005850 comp rx=0 tx=0).stop 2024-09-10T23:00:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 msgr2=0x7fa65c19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d950 tx=0x7fa64c00de10 comp rx=0 tx=0).stop 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 shutdown_connections 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa6400b5580 0x7fa6400b7a40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa65c10f220 0x7fa65c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa65c075de0 0x7fa65c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 --2- 172.21.15.67:0/349216664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa65c0754c0 0x7fa65c19b130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 >> 172.21.15.67:0/349216664 conn(0x7fa65c0fe660 msgr2=0x7fa65c1113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 shutdown_connections 2024-09-10T23:00:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:04.815+0000 7fa661200640 1 -- 172.21.15.67:0/349216664 wait complete. 2024-09-10T23:00:05.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:04 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/349216664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:05.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:04 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/349216664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:05.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:04 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/349216664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:05.650 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T23:00:05.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T23:00:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:06.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:05 smithi067 ceph-mon[30939]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:05 smithi092 ceph-mon[34223]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:05 smithi145 ceph-mon[34397]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.839+0000 7f1f56e00640 1 -- 172.21.15.67:0/3627233750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f501087f0 msgr2=0x7f1f50108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.839+0000 7f1f56e00640 1 --2- 172.21.15.67:0/3627233750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f501087f0 0x7f1f50108c30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009900 tx=0x7f1f3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.840+0000 7f1f56e00640 1 -- 172.21.15.67:0/3627233750 shutdown_connections 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.840+0000 7f1f56e00640 1 --2- 172.21.15.67:0/3627233750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f50109170 0x7f1f5010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.840+0000 7f1f56e00640 1 --2- 172.21.15.67:0/3627233750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f501087f0 0x7f1f50108c30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.840+0000 7f1f56e00640 1 --2- 172.21.15.67:0/3627233750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f50102de0 0x7f1f501031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.840+0000 7f1f56e00640 1 -- 172.21.15.67:0/3627233750 >> 172.21.15.67:0/3627233750 conn(0x7f1f500fe830 msgr2=0x7f1f50100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.841+0000 7f1f56e00640 1 -- 172.21.15.67:0/3627233750 shutdown_connections 2024-09-10T23:00:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.841+0000 7f1f56e00640 1 -- 172.21.15.67:0/3627233750 wait complete. 2024-09-10T23:00:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 Processor -- start 2024-09-10T23:00:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 -- start start 2024-09-10T23:00:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f50102de0 0x7f1f5019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 0x7f1f501a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50112cc0 con 0x7f1f50109170 2024-09-10T23:00:06.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50112b40 con 0x7f1f50102de0 2024-09-10T23:00:06.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.843+0000 7f1f56e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50112e40 con 0x7f1f501087f0 2024-09-10T23:00:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.844+0000 7f1f54e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 0x7f1f501a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.844+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.844+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:49648/0 (socket says 172.21.15.67:49648) 2024-09-10T23:00:06.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.844+0000 7f1f4f400640 1 -- 172.21.15.67:0/168862317 learned_addr learned my addr 172.21.15.67:0/168862317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:06.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.844+0000 7f1f54e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 0x7f1f501a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51814/0 (socket says 172.21.15.67:51814) 2024-09-10T23:00:06.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 -- 172.21.15.67:0/168862317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f50102de0 msgr2=0x7f1f5019b470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:00:06.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f50102de0 0x7f1f5019b470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 -- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 msgr2=0x7f1f501a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 0x7f1f501a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 -- 172.21.15.67:0/168862317 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f501a1520 con 0x7f1f501087f0 2024-09-10T23:00:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.845+0000 7f1f4f400640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c02a910 tx=0x7f1f3c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.846+0000 7f1f4cc00640 1 -- 172.21.15.67:0/168862317 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c03f070 con 0x7f1f501087f0 2024-09-10T23:00:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.846+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f501a17b0 con 0x7f1f501087f0 2024-09-10T23:00:06.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.846+0000 7f1f4cc00640 1 -- 172.21.15.67:0/168862317 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f3c03baf0 con 0x7f1f501087f0 2024-09-10T23:00:06.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.846+0000 7f1f4cc00640 1 -- 172.21.15.67:0/168862317 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c035030 con 0x7f1f501087f0 2024-09-10T23:00:06.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.847+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f50109840 con 0x7f1f501087f0 2024-09-10T23:00:06.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 Processor -- start 2024-09-10T23:00:06.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 -- start start 2024-09-10T23:00:06.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944aa0 0x56537f9504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f952e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56537f8b9ca0 con 0x56537f944180 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56537f8a90d0 con 0x56537f950a30 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.852+0000 7f1f5985e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56537f8acee0 con 0x56537f944aa0 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.853+0000 7f1f54e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.853+0000 7f1f54e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51822/0 (socket says 172.21.15.67:51822) 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.853+0000 7f1f54e00640 1 -- 172.21.15.67:0/3114462845 learned_addr learned my addr 172.21.15.67:0/3114462845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.854+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944aa0 0x56537f9504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.854+0000 7f1f4cc00640 1 -- 172.21.15.67:0/168862317 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1f3c03b660 con 0x7f1f501087f0 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.854+0000 7f1f4f400640 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f952e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 -- 172.21.15.67:0/3114462845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944aa0 msgr2=0x56537f9504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944aa0 0x56537f9504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 -- 172.21.15.67:0/3114462845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 msgr2=0x56537f952e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f952e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 -- 172.21.15.67:0/3114462845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537f85b400 con 0x56537f944180 2024-09-10T23:00:06.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.855+0000 7f1f54e00640 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1f48015d10 tx=0x7f1f48009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470b7341e54b668d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.856+0000 7f1f45600640 1 -- 172.21.15.67:0/3114462845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48025070 con 0x56537f944180 2024-09-10T23:00:06.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.856+0000 7f1f45600640 1 -- 172.21.15.67:0/3114462845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f48003970 con 0x56537f944180 2024-09-10T23:00:06.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.856+0000 7f1f45600640 1 -- 172.21.15.67:0/3114462845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48004540 con 0x56537f944180 2024-09-10T23:00:06.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.857+0000 7f1f4cc00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 0x7f1f340b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.857+0000 7f1f4cc00640 1 -- 172.21.15.67:0/168862317 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f1f3c1068f0 con 0x7f1f501087f0 2024-09-10T23:00:06.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.858+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 0x7f1f340b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.858+0000 7f1f5985e740 1 -- 172.21.15.67:0/3114462845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 msgr2=0x56537f944560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.858+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1f48015d10 tx=0x7f1f48009500 comp rx=0 tx=0).stop 2024-09-10T23:00:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f5985e740 1 -- 172.21.15.67:0/3114462845 shutdown_connections 2024-09-10T23:00:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f952e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944aa0 0x56537f9504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3114462845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944180 0x56537f944560 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f5985e740 1 -- 172.21.15.67:0/3114462845 >> 172.21.15.67:0/3114462845 conn(0x56537f941fb0 msgr2=0x56537f943150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.859+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 0x7f1f340b7910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56537f942e90 tx=0x7f1f40005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.860+0000 7f1f5985e740 1 -- 172.21.15.67:0/3114462845 shutdown_connections 2024-09-10T23:00:06.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.860+0000 7f1f5985e740 1 -- 172.21.15.67:0/3114462845 wait complete. 2024-09-10T23:00:06.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.861+0000 7f1f5985e740 1 -- start start 2024-09-10T23:00:06.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.862+0000 7f1f5985e740 1 Processor -- start 2024-09-10T23:00:06.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.863+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f950360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.863+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.863+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f94de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51830/0 (socket says 172.21.15.67:51830) 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f4fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f950360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f4f400640 1 -- 172.21.15.67:0/844865099 learned_addr learned my addr 172.21.15.67:0/844865099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56537f70a7a0 con 0x56537f944aa0 2024-09-10T23:00:06.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56537f84ea10 con 0x56537f950a30 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.864+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56537f9536b0 con 0x56537f944180 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f54e00640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f94de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f5985e740 10 client.0 _ll_register_callbacks cb 0x56537f85ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 -- 172.21.15.67:0/844865099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 msgr2=0x56537f950360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f950360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 -- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 msgr2=0x56537f94de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f94de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537f616b90 con 0x56537f944aa0 2024-09-10T23:00:06.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f4f400640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c8f50 tx=0x7f1f3c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c941bbfcd0406f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.865+0000 7f1f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.866+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0cc2d0 con 0x56537f944aa0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.866+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f1f3c103ed0 con 0x56537f944aa0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.866+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f48003970 con 0x56537f944aa0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.867+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c0c80f0 con 0x56537f944aa0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.867+0000 7f1f5985e740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.867+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56537f95f040 con 0x56537f944aa0 2024-09-10T23:00:06.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f48003970 con 0x56537f944aa0 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f3c106ce0 con 0x56537f944aa0 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 10 client.15672 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 20 client.15672 populate_metadata read hostname 'smithi067' 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f960480 0x56537f962920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 4 client.15672 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:06.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.868+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537f8bbbf0 con 0x56537f960480 2024-09-10T23:00:06.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.869+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f960480 0x56537f962920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.870+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f960480 0x56537f962920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c836f229fe6308 server_cookie=fd862d1cf1e589c in_seq=0 out_seq=0 2024-09-10T23:00:06.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.870+0000 7f1f3a000640 10 client.15672 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:06.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f3a000640 1 -- 172.21.15.67:0/844865099 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56537f8bbbf0 con 0x56537f960480 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f3a000640 10 client.15672 handle_command_reply: tid=0 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 client.15672 shutdown 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 msgr2=0x56537f94d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c8f50 tx=0x7f1f3c005be0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 shutdown_connections 2024-09-10T23:00:06.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f960480 0x56537f962920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f94de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f944aa0 0x56537f94d7b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 --2- 172.21.15.67:0/844865099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f950360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f3a000640 0 client.15672 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.871+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 >> 172.21.15.67:0/844865099 conn(0x56537f941fb0 msgr2=0x56537f947b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.872+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 shutdown_connections 2024-09-10T23:00:06.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.872+0000 7f1f5985e740 1 -- 172.21.15.67:0/844865099 wait complete. 2024-09-10T23:00:06.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.872+0000 7f1f5985e740 20 client.15672 trim_cache size 0 max 16384 2024-09-10T23:00:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.909+0000 7f1f5985e740 1 Processor -- start 2024-09-10T23:00:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.909+0000 7f1f5985e740 1 -- start start 2024-09-10T23:00:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f94f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 0x56537f952360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501b05b0 con 0x56537f950a30 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501ade80 con 0x56537f9434d0 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.910+0000 7f1f5985e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501abcd0 con 0x56537f944180 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.911+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.911+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51838/0 (socket says 172.21.15.67:51838) 2024-09-10T23:00:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.911+0000 7f1f4f400640 1 -- 172.21.15.67:0/1638012770 learned_addr learned my addr 172.21.15.67:0/1638012770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.911+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f94f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.911+0000 7f1f54e00640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 0x56537f952360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 -- 172.21.15.67:0/1638012770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 msgr2=0x56537f94f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f94f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 -- 172.21.15.67:0/1638012770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 msgr2=0x56537f952360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 0x56537f952360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 -- 172.21.15.67:0/1638012770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537f95cd80 con 0x56537f950a30 2024-09-10T23:00:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f54e00640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 0x56537f952360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.912+0000 7f1f4f400640 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0cc910 tx=0x7f1f3c03beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4321ad54236f61bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.913+0000 7f1f44c00640 1 -- 172.21.15.67:0/1638012770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0cc0f0 con 0x56537f950a30 2024-09-10T23:00:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.913+0000 7f1f44c00640 1 -- 172.21.15.67:0/1638012770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f3c0cc490 con 0x56537f950a30 2024-09-10T23:00:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.913+0000 7f1f44c00640 1 -- 172.21.15.67:0/1638012770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0cc0f0 con 0x56537f950a30 2024-09-10T23:00:06.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 -- 172.21.15.67:0/1638012770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 msgr2=0x56537f951e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0cc910 tx=0x7f1f3c03beb0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 -- 172.21.15.67:0/1638012770 shutdown_connections 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f9434d0 0x56537f952360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f950a30 0x56537f951e20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 --2- 172.21.15.67:0/1638012770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f94f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.914+0000 7f1f5985e740 1 -- 172.21.15.67:0/1638012770 >> 172.21.15.67:0/1638012770 conn(0x56537f941fb0 msgr2=0x56537f94ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.915+0000 7f1f5985e740 1 -- 172.21.15.67:0/1638012770 shutdown_connections 2024-09-10T23:00:06.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.915+0000 7f1f5985e740 1 -- 172.21.15.67:0/1638012770 wait complete. 2024-09-10T23:00:06.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.916+0000 7f1f5985e740 1 -- start start 2024-09-10T23:00:06.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.916+0000 7f1f5985e740 1 Processor -- start 2024-09-10T23:00:06.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f9589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f958ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 0x56537f955970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501ac340 con 0x56537f959420 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f958ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f4f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f958ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:54686/0 (socket says 172.21.15.67:54686) 2024-09-10T23:00:06.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f4f400640 1 -- 172.21.15.67:0/3368570546 learned_addr learned my addr 172.21.15.67:0/3368570546 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:06.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501ad7b0 con 0x56537f950a30 2024-09-10T23:00:06.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501aca10 con 0x56537f944180 2024-09-10T23:00:06.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.917+0000 7f1f5985e740 10 client.0 _ll_register_callbacks cb 0x7f1f3c0c8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:06.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f9589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f54e00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 0x56537f955970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 -- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 msgr2=0x56537f958ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f958ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 -- 172.21.15.67:0/3368570546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 msgr2=0x56537f955970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 0x56537f955970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.918+0000 7f1f4fe00640 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537f941de0 con 0x56537f944180 2024-09-10T23:00:06.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.919+0000 7f1f54e00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 0x56537f955970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.919+0000 7f1f4fe00640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f9589a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x56537f962f20 tx=0x7f1f40004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6847fa3b79dc7ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.919+0000 7f1f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:00:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.920+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40034060 con 0x56537f944180 2024-09-10T23:00:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.920+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f1f4001e830 con 0x56537f944180 2024-09-10T23:00:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.920+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c0cc490 con 0x56537f944180 2024-09-10T23:00:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.920+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f4001ea70 con 0x56537f944180 2024-09-10T23:00:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f5985e740 10 client.25408 fetch_fsmap learned FSMap version 27 2024-09-10T23:00:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56537f9cfb10 con 0x56537f944180 2024-09-10T23:00:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f3c0cc490 con 0x56537f944180 2024-09-10T23:00:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1f4002fbc0 con 0x56537f944180 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f5985e740 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f5985e740 10 client.25408 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.921+0000 7f1f5985e740 20 client.25408 populate_metadata read hostname 'smithi067' 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.922+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f9d0290 0x56537f9d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.922+0000 7f1f5985e740 4 client.25408 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T23:00:06.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.922+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x56537f8b76a0 con 0x56537f9d0290 2024-09-10T23:00:06.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.922+0000 7f1f4f400640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f9d0290 0x56537f9d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:06.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.923+0000 7f1f4f400640 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f9d0290 0x56537f9d26b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a19d6a5e467028 server_cookie=5742446fa22e36d5 in_seq=0 out_seq=0 2024-09-10T23:00:06.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.923+0000 7f1f3a000640 10 client.25408 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 27, 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 2, 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "eb7e4854": { 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 2, 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:00:06.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.2, 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:06.945 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f3a000640 1 -- 172.21.15.67:0/3368570546 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56537f8b76a0 con 0x56537f9d0290 2024-09-10T23:00:06.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f3a000640 10 client.25408 handle_command_reply: tid=0 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 client.25408 shutdown 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 msgr2=0x56537f9589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f9589a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x56537f962f20 tx=0x7f1f40004730 comp rx=0 tx=0).stop 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 shutdown_connections 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56537f9d0290 0x56537f9d26b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56537f959420 0x56537f955970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56537f950a30 0x56537f958ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 --2- 172.21.15.67:0/3368570546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537f944180 0x56537f9589a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 >> 172.21.15.67:0/3368570546 conn(0x56537f941fb0 msgr2=0x56537f94f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.938+0000 7f1f3a000640 0 client.25408 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.939+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 shutdown_connections 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.939+0000 7f1f5985e740 1 -- 172.21.15.67:0/3368570546 wait complete. 2024-09-10T23:00:06.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.939+0000 7f1f5985e740 20 client.25408 trim_cache size 0 max 16384 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 msgr2=0x7f1f340b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 0x7f1f340b7910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56537f942e90 tx=0x7f1f40005f90 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 msgr2=0x7f1f5019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c02a910 tx=0x7f1f3c003b90 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 shutdown_connections 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1f340b5450 0x7f1f340b7910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f50109170 0x7f1f501a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f501087f0 0x7f1f5019b9b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 --2- 172.21.15.67:0/168862317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f50102de0 0x7f1f5019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.941+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 >> 172.21.15.67:0/168862317 conn(0x7f1f500fe830 msgr2=0x7f1f50105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.942+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 shutdown_connections 2024-09-10T23:00:06.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:06.942+0000 7f1f56e00640 1 -- 172.21.15.67:0/168862317 wait complete. 2024-09-10T23:00:08.006 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eb7e4854', quiesce took 4.6 seconds. Will release after: 20.6 2024-09-10T23:00:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:07 smithi092 ceph-mon[34223]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-10T23:00:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:07 smithi067 ceph-mon[30939]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-10T23:00:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:07 smithi145 ceph-mon[34397]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-10T23:00:10.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:09 smithi092 ceph-mon[34223]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:09 smithi067 ceph-mon[30939]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:09 smithi145 ceph-mon[34397]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-10T23:00:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:11 smithi145 ceph-mon[34397]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:11 smithi092 ceph-mon[34223]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:11 smithi067 ceph-mon[30939]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:13 smithi092 ceph-mon[34223]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:13 smithi067 ceph-mon[30939]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:13 smithi145 ceph-mon[34397]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-10T23:00:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:15 smithi092 ceph-mon[34223]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-10T23:00:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:15 smithi067 ceph-mon[30939]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-10T23:00:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:15 smithi145 ceph-mon[34397]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-10T23:00:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:17 smithi092 ceph-mon[34223]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T23:00:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:17 smithi067 ceph-mon[30939]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T23:00:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:17 smithi145 ceph-mon[34397]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T23:00:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:19 smithi092 ceph-mon[34223]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:00:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:19 smithi067 ceph-mon[30939]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:00:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:19 smithi145 ceph-mon[34397]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:00:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:21 smithi092 ceph-mon[34223]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:21 smithi067 ceph-mon[30939]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:21 smithi145 ceph-mon[34397]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:23 smithi092 ceph-mon[34223]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:23 smithi067 ceph-mon[30939]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:23 smithi145 ceph-mon[34397]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:00:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:25 smithi092 ceph-mon[34223]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:25 smithi067 ceph-mon[30939]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:25 smithi145 ceph-mon[34397]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:27 smithi092 ceph-mon[34223]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:28.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:27 smithi067 ceph-mon[30939]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:27 smithi145 ceph-mon[34397]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:28.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:00:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 -- 172.21.15.67:0/2806098297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba640754e0 msgr2=0x7fba640758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 --2- 172.21.15.67:0/2806098297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba640754e0 0x7fba640758c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fba54009900 tx=0x7fba5402a410 comp rx=0 tx=0).stop 2024-09-10T23:00:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 -- 172.21.15.67:0/2806098297 shutdown_connections 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 --2- 172.21.15.67:0/2806098297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba6410f200 0x7fba641115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 --2- 172.21.15.67:0/2806098297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64075e00 0x7fba6410ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 --2- 172.21.15.67:0/2806098297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba640754e0 0x7fba640758c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 -- 172.21.15.67:0/2806098297 >> 172.21.15.67:0/2806098297 conn(0x7fba640fe660 msgr2=0x7fba64100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 -- 172.21.15.67:0/2806098297 shutdown_connections 2024-09-10T23:00:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.741+0000 7fba69600640 1 -- 172.21.15.67:0/2806098297 wait complete. 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.742+0000 7fba69600640 1 Processor -- start 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.742+0000 7fba69600640 1 -- start start 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 0x7fba6419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba6410f200 0x7fba641a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba64114ce0 con 0x7fba64075e00 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba64114b60 con 0x7fba6410f200 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba69600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba64114e60 con 0x7fba640754e0 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba63e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba63e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:42002/0 (socket says 172.21.15.67:42002) 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba63e00640 1 -- 172.21.15.67:0/1130396285 learned_addr learned my addr 172.21.15.67:0/1130396285 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.743+0000 7fba68c00640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba6410f200 0x7fba641a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63400640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 0x7fba6419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 -- 172.21.15.67:0/1130396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba6410f200 msgr2=0x7fba641a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba6410f200 0x7fba641a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 -- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 msgr2=0x7fba6419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 0x7fba6419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 -- 172.21.15.67:0/1130396285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641a1260 con 0x7fba640754e0 2024-09-10T23:00:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63400640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 0x7fba6419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.744+0000 7fba63e00640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba5402a930 tx=0x7fba54003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.745+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5403f070 con 0x7fba640754e0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.745+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba5403baf0 con 0x7fba640754e0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.745+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba641a14f0 con 0x7fba640754e0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.745+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54035030 con 0x7fba640754e0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.745+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641a7c10 con 0x7fba640754e0 2024-09-10T23:00:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.746+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba640758c0 con 0x7fba640754e0 2024-09-10T23:00:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.748+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fba5403b660 con 0x7fba640754e0 2024-09-10T23:00:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.750+0000 7fba60c00640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 0x7fba480b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.750+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fba541070b0 con 0x7fba640754e0 2024-09-10T23:00:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.750+0000 7fba63400640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 0x7fba480b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.751+0000 7fba63400640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 0x7fba480b7a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba64100e00 tx=0x7fba50005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:29.752+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fba540cd840 con 0x7fba640754e0 2024-09-10T23:00:29.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:29 smithi067 ceph-mon[30939]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:30.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.052+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba6419c2e0 con 0x7fba640754e0 2024-09-10T23:00:30.055 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:00:30.055 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":27,"btime":"2024-09-10T22:59:24:334283+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T22:59:23.805517+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:00:30.056 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 27 2024-09-10T23:00:30.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.053+0000 7fba60c00640 1 -- 172.21.15.67:0/1130396285 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11903 (secure 0 0 0) 0x7fba540cd140 con 0x7fba640754e0 2024-09-10T23:00:30.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 msgr2=0x7fba480b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 0x7fba480b7a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba64100e00 tx=0x7fba50005f90 comp rx=0 tx=0).stop 2024-09-10T23:00:30.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 msgr2=0x7fba6419b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba5402a930 tx=0x7fba54003b90 comp rx=0 tx=0).stop 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 shutdown_connections 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba480b5580 0x7fba480b7a40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba6410f200 0x7fba641a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba64075e00 0x7fba6419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 --2- 172.21.15.67:0/1130396285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba640754e0 0x7fba6419b1b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 >> 172.21.15.67:0/1130396285 conn(0x7fba640fe660 msgr2=0x7fba64111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 shutdown_connections 2024-09-10T23:00:30.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:30.056+0000 7fba69600640 1 -- 172.21.15.67:0/1130396285 wait complete. 2024-09-10T23:00:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:29 smithi092 ceph-mon[34223]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:29 smithi145 ceph-mon[34397]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:31.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:30 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1130396285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:31.197 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id eb7e4854 --release --await' 2024-09-10T23:00:31.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id eb7e4854 --release --await 2024-09-10T23:00:31.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:30 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1130396285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:31.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:30 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1130396285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:31.624 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:31 smithi067 ceph-mon[30939]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:31 smithi092 ceph-mon[34223]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:32.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1376670966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b0103b10 msgr2=0x7fa9b0103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1376670966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b0103b10 0x7fa9b0103ef0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009900 tx=0x7fa9a402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1376670966 shutdown_connections 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1376670966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b010cf70 0x7fa9b010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1376670966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1376670966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b0103b10 0x7fa9b0103ef0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.298+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1376670966 >> 172.21.15.67:0/1376670966 conn(0x7fa9b00fd5b0 msgr2=0x7fa9b00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.299+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1376670966 shutdown_connections 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.299+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1376670966 wait complete. 2024-09-10T23:00:32.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.300+0000 7fa9b6c00640 1 Processor -- start 2024-09-10T23:00:32.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.300+0000 7fa9b6c00640 1 -- start start 2024-09-10T23:00:32.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b0103b10 0x7fa9b01017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b0101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0112ae0 con 0x7fa9b010cf70 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0112960 con 0x7fa9b0104430 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0112c60 con 0x7fa9b0103b10 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.301+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b0101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35032/0 (socket says 172.21.15.67:35032) 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b0101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:44872/0 (socket says 172.21.15.67:44872) 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 -- 172.21.15.67:0/1093754425 learned_addr learned my addr 172.21.15.67:0/1093754425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b5800640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b0103b10 0x7fa9b01017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 -- 172.21.15.67:0/1093754425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b0103b10 msgr2=0x7fa9b01017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b0103b10 0x7fa9b01017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 -- 172.21.15.67:0/1093754425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 msgr2=0x7fa9b0101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b0101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.302+0000 7fa9b6200640 1 -- 172.21.15.67:0/1093754425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a7350 con 0x7fa9b010cf70 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9b6200640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800eb10 tx=0x7fa9a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9ae000640 1 -- 172.21.15.67:0/1093754425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8004b20 con 0x7fa9b010cf70 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a7640 con 0x7fa9b010cf70 2024-09-10T23:00:32.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a7bd0 con 0x7fa9b010cf70 2024-09-10T23:00:32.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9ae000640 1 -- 172.21.15.67:0/1093754425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9a8004cc0 con 0x7fa9b010cf70 2024-09-10T23:00:32.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.303+0000 7fa9ae000640 1 -- 172.21.15.67:0/1093754425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8021440 con 0x7fa9b010cf70 2024-09-10T23:00:32.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.307+0000 7fa9bade3740 1 Processor -- start 2024-09-10T23:00:32.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.308+0000 7fa9ae000640 1 -- 172.21.15.67:0/1093754425 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa9a80215e0 con 0x7fa9b010cf70 2024-09-10T23:00:32.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.308+0000 7fa9bade3740 1 -- start start 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x5633232845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x563323292e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5633231fb290 con 0x563323284af0 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633231ea6c0 con 0x5633232841d0 2024-09-10T23:00:32.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.309+0000 7fa9bade3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633231ee4d0 con 0x563323290a80 2024-09-10T23:00:32.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x5633232845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35048/0 (socket says 172.21.15.67:35048) 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9b5800640 1 -- 172.21.15.67:0/33436228 learned_addr learned my addr 172.21.15.67:0/33436228 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x563323292e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9ae000640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 0x7fa9980c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.310+0000 7fa9ae000640 1 -- 172.21.15.67:0/1093754425 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa9a80e6860 con 0x7fa9b010cf70 2024-09-10T23:00:32.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.311+0000 7fa9b6200640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 0x7fa9980c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.311+0000 7fa9b4e00640 1 -- 172.21.15.67:0/33436228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 msgr2=0x563323292e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.311+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x563323292e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.311+0000 7fa9b4e00640 1 -- 172.21.15.67:0/33436228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 msgr2=0x563323290540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.311+0000 7fa9b6200640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 0x7fa9980c0250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80e3c20 tx=0x7fa9a8009200 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.312+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.312+0000 7fa9b4e00640 1 -- 172.21.15.67:0/33436228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563323177460 con 0x5633232841d0 2024-09-10T23:00:32.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.312+0000 7fa9b5800640 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:32.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.312+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x5633232845b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9b01032b0 tx=0x7fa99c009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d717407f12bd0ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.313+0000 7fa9a2a00640 1 -- 172.21.15.67:0/33436228 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c004030 con 0x5633232841d0 2024-09-10T23:00:32.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.313+0000 7fa9a2a00640 1 -- 172.21.15.67:0/33436228 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c005020 con 0x5633232841d0 2024-09-10T23:00:32.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.313+0000 7fa9a2a00640 1 -- 172.21.15.67:0/33436228 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c004bc0 con 0x5633232841d0 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 -- 172.21.15.67:0/33436228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 msgr2=0x5633232845b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x5633232845b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9b01032b0 tx=0x7fa99c009610 comp rx=0 tx=0).stop 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 -- 172.21.15.67:0/33436228 shutdown_connections 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x563323292e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323290540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 --2- 172.21.15.67:0/33436228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x5633232845b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 -- 172.21.15.67:0/33436228 >> 172.21.15.67:0/33436228 conn(0x563323282000 msgr2=0x5633232831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 -- 172.21.15.67:0/33436228 shutdown_connections 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.314+0000 7fa9bade3740 1 -- 172.21.15.67:0/33436228 wait complete. 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.315+0000 7fa9bade3740 1 -- start start 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.315+0000 7fa9bade3740 1 Processor -- start 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x563323290540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x56332328e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56332304bbb0 con 0x563323284af0 2024-09-10T23:00:32.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633231856c0 con 0x5633232841d0 2024-09-10T23:00:32.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5633232937d0 con 0x563323290a80 2024-09-10T23:00:32.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9bade3740 10 client.0 _ll_register_callbacks cb 0x5633231a0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:32.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9b5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35054/0 (socket says 172.21.15.67:35054) 2024-09-10T23:00:32.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9b5800640 1 -- 172.21.15.67:0/1115119654 learned_addr learned my addr 172.21.15.67:0/1115119654 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:32.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x563323290540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.316+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x56332328e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 -- 172.21.15.67:0/1115119654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 msgr2=0x56332328e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x56332328e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 -- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 msgr2=0x563323290540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x563323290540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 -- 172.21.15.67:0/1115119654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563322f48b00 con 0x563323284af0 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x563323290540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa9b5800640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x563323287350 tx=0x7fa9a40044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9a4d4c3a28dc9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.317+0000 7fa997400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:00:32.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4035040 con 0x563323284af0 2024-09-10T23:00:32.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa9a4005590 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99c005020 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9a40046c0 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa9bade3740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.318+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563322fbf960 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c005020 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa9a4003a80 con 0x563323284af0 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 10 client.15693 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 20 client.15693 populate_metadata read hostname 'smithi067' 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5633232a0690 0x5633232a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 4 client.15693 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.319+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633231fd1e0 con 0x5633232a0690 2024-09-10T23:00:32.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.320+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5633232a0690 0x5633232a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.321+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5633232a0690 0x5633232a2af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f18d7c133a0410 server_cookie=3c39b01c6dcc3ed0 in_seq=0 out_seq=0 2024-09-10T23:00:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.321+0000 7fa997400640 10 client.15693 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.321+0000 7fa997400640 1 -- 172.21.15.67:0/1115119654 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633231fd1e0 con 0x5633232a0690 2024-09-10T23:00:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.321+0000 7fa997400640 10 client.15693 handle_command_reply: tid=0 2024-09-10T23:00:32.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 client.15693 shutdown 2024-09-10T23:00:32.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 msgr2=0x56332328d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x563323287350 tx=0x7fa9a40044f0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 shutdown_connections 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5633232a0690 0x5633232a2af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563323290a80 0x56332328e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x56332328d9d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 --2- 172.21.15.67:0/1115119654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633232841d0 0x563323290540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 >> 172.21.15.67:0/1115119654 conn(0x563323282000 msgr2=0x563323287ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.322+0000 7fa997400640 0 client.15693 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.323+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 shutdown_connections 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.323+0000 7fa9bade3740 1 -- 172.21.15.67:0/1115119654 wait complete. 2024-09-10T23:00:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.323+0000 7fa9bade3740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T23:00:32.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.341+0000 7fa9bade3740 1 Processor -- start 2024-09-10T23:00:32.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.341+0000 7fa9bade3740 1 -- start start 2024-09-10T23:00:32.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x56332328adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 0x56332328b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 0x563323282f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa990012810 con 0x563323290a80 2024-09-10T23:00:32.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9900100e0 con 0x563323284af0 2024-09-10T23:00:32.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9bade3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000df30 con 0x5633232841d0 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 0x563323282f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 0x563323282f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35062/0 (socket says 172.21.15.67:35062) 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9b4e00640 1 -- 172.21.15.67:0/4011949273 learned_addr learned my addr 172.21.15.67:0/4011949273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x56332328adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.342+0000 7fa9b5800640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 0x56332328b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 -- 172.21.15.67:0/4011949273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 msgr2=0x56332328b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 0x56332328b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 -- 172.21.15.67:0/4011949273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 msgr2=0x563323282f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 0x563323282f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 -- 172.21.15.67:0/4011949273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56332328e4d0 con 0x5633232841d0 2024-09-10T23:00:32.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b5800640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 0x56332328b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:32.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x56332328adf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0095a0 tx=0x7fa99c009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9f5c5af253f460 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.343+0000 7fa9a2000640 1 -- 172.21.15.67:0/4011949273 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c004d90 con 0x5633232841d0 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9a2000640 1 -- 172.21.15.67:0/4011949273 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c004f30 con 0x5633232841d0 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9a2000640 1 -- 172.21.15.67:0/4011949273 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c01e460 con 0x5633232841d0 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 -- 172.21.15.67:0/4011949273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 msgr2=0x56332328adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x56332328adf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0095a0 tx=0x7fa99c009530 comp rx=0 tx=0).stop 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 -- 172.21.15.67:0/4011949273 shutdown_connections 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323290a80 0x563323282f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323284af0 0x56332328b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 --2- 172.21.15.67:0/4011949273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x56332328adf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.344+0000 7fa9bade3740 1 -- 172.21.15.67:0/4011949273 >> 172.21.15.67:0/4011949273 conn(0x563323282000 msgr2=0x56332328c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.345+0000 7fa9bade3740 1 -- 172.21.15.67:0/4011949273 shutdown_connections 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.345+0000 7fa9bade3740 1 -- 172.21.15.67:0/4011949273 wait complete. 2024-09-10T23:00:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.345+0000 7fa9bade3740 1 -- start start 2024-09-10T23:00:32.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.345+0000 7fa9bade3740 1 Processor -- start 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323299040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323290a80 0x563323293ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000e5a0 con 0x563323284af0 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000fa10 con 0x563323290a80 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa99000ec70 con 0x5633232841d0 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9bade3740 10 client.0 _ll_register_callbacks cb 0x7fa9a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9b4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:42062/0 (socket says 172.21.15.67:42062) 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.346+0000 7fa9b4e00640 1 -- 172.21.15.67:0/70594306 learned_addr learned my addr 172.21.15.67:0/70594306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323290a80 0x563323293ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 -- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323290a80 msgr2=0x563323293ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:32.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323290a80 0x563323293ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 -- 172.21.15.67:0/70594306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 msgr2=0x563323299040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:32.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b5800640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323299040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323299040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b4e00640 1 -- 172.21.15.67:0/70594306 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563323284920 con 0x5633232841d0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.347+0000 7fa9b5800640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323299040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.348+0000 7fa9b4e00640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa99c004830 tx=0x7fa99c021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1513bfd0b2ddbcb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.348+0000 7fa997400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.348+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c036680 con 0x5633232841d0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.348+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fa99c036820 con 0x5633232841d0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.348+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99c004f30 con 0x5633232841d0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.349+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa99c04a050 con 0x5633232841d0 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.349+0000 7fa9bade3740 10 client.25429 fetch_fsmap learned FSMap version 27 2024-09-10T23:00:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.349+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56332330dc40 con 0x5633232841d0 2024-09-10T23:00:32.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.349+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c004f30 con 0x5633232841d0 2024-09-10T23:00:32.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.349+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa99c01eb80 con 0x5633232841d0 2024-09-10T23:00:32.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 10 client.25429 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 20 client.25429 populate_metadata read hostname 'smithi067' 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56332330e0b0 0x5633233104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 4 client.25429 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eb7e4854", "release": true, "await": true}] 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eb7e4854", "release": true, "await": true}) -- 0x5633231f8c90 con 0x56332330e0b0 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.350+0000 7fa9b5800640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56332330e0b0 0x5633233104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.351+0000 7fa9b5800640 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56332330e0b0 0x5633233104a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80713f2b9411d45e server_cookie=b1d24ae0031d0857 in_seq=0 out_seq=0 2024-09-10T23:00:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.351+0000 7fa997400640 10 client.25429 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa997400640 1 -- 172.21.15.67:0/70594306 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5633231f8c90 con 0x56332330e0b0 2024-09-10T23:00:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa997400640 10 client.25429 handle_command_reply: tid=0 2024-09-10T23:00:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 client.25429 shutdown 2024-09-10T23:00:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 msgr2=0x563323298b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa99c004830 tx=0x7fa99c021040 comp rx=0 tx=0).stop 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 shutdown_connections 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56332330e0b0 0x5633233104a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563323290a80 0x563323293ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563323284af0 0x563323299040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 --2- 172.21.15.67:0/70594306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5633232841d0 0x563323298b00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.353+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 >> 172.21.15.67:0/70594306 conn(0x563323282000 msgr2=0x56332328adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.354+0000 7fa997400640 0 client.25429 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.354+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 shutdown_connections 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.354+0000 7fa9bade3740 1 -- 172.21.15.67:0/70594306 wait complete. 2024-09-10T23:00:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.354+0000 7fa9bade3740 20 client.25429 trim_cache size 0 max 16384 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 27, 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 4, 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "eb7e4854": { 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 4, 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:00:32.357 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.2, 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:32.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 msgr2=0x7fa9980c0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 0x7fa9980c0250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80e3c20 tx=0x7fa9a8009200 comp rx=0 tx=0).stop 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 msgr2=0x7fa9b0102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800eb10 tx=0x7fa9a800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 shutdown_connections 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9980bdd90 0x7fa9980c0250 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9b010cf70 0x7fa9b0102240 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b0104430 0x7fa9b0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 --2- 172.21.15.67:0/1093754425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9b0103b10 0x7fa9b01017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.355+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 >> 172.21.15.67:0/1093754425 conn(0x7fa9b00fd5b0 msgr2=0x7fa9b010f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.356+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 shutdown_connections 2024-09-10T23:00:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:00:32.356+0000 7fa9b6c00640 1 -- 172.21.15.67:0/1093754425 wait complete. 2024-09-10T23:00:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:31 smithi145 ceph-mon[34397]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:33.395 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eb7e4854', total seconds elapsed: 30.0 2024-09-10T23:00:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:33 smithi092 ceph-mon[34223]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:33 smithi067 ceph-mon[30939]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:33 smithi145 ceph-mon[34397]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:35 smithi092 ceph-mon[34223]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:35 smithi067 ceph-mon[30939]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:35 smithi145 ceph-mon[34397]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:00:37.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:37.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:37.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:37.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:36 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:00:37.053 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:37.053 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:37.053 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:37.053 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:36 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:00:37.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:37.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:37.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:37.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:36 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:00:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:37 smithi092 ceph-mon[34223]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:37 smithi067 ceph-mon[30939]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:38.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:37 smithi145 ceph-mon[34397]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:00:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:39 smithi092 ceph-mon[34223]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:39 smithi067 ceph-mon[30939]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:39 smithi145 ceph-mon[34397]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:00:40.761 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 63% (7719/12088) Updating files: 64% (7737/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-10T23:00:42.162 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.163 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-09-10T23:00:42.164 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.164 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T23:00:42.164 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-09-10T23:00:42.164 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:00:42.176 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T23:00:42.239 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T23:00:42.244 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T23:00:42.244 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T23:00:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:41 smithi092 ceph-mon[34223]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:41 smithi067 ceph-mon[30939]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:42.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:41 smithi145 ceph-mon[34397]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:00:42.966 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T23:00:43.056 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T23:00:43.123 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T23:00:43.126 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T23:00:43.126 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T23:00:43.191 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T23:00:43.198 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T23:00:43.198 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T23:00:43.234 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-10T23:00:43.236 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-10T23:00:43.236 DEBUG:teuthology.orchestra.run.smithi067:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T23:00:43.338 INFO:tasks.workunit.client.0.smithi067.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T23:00:43.340 INFO:tasks.workunit.client.0.smithi067.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-10T23:00:43.340 INFO:tasks.workunit.client.0.smithi067.stderr:+ trap -- retry EXIT 2024-09-10T23:00:43.340 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rf ffsb 2024-09-10T23:00:43.343 INFO:tasks.workunit.client.0.smithi067.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-10T23:00:43.352 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into 'ffsb'... 2024-09-10T23:00:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:43 smithi092 ceph-mon[34223]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:43 smithi067 ceph-mon[30939]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:43 smithi145 ceph-mon[34397]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:00:45.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ trap - EXIT 2024-09-10T23:00:45.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ffsb 2024-09-10T23:00:45.940 INFO:tasks.workunit.client.0.smithi067.stderr:+ ./configure 2024-09-10T23:00:46.213 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T23:00:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:45 smithi092 ceph-mon[34223]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T23:00:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:45 smithi067 ceph-mon[30939]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T23:00:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:45 smithi145 ceph-mon[34397]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T23:00:47.230 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether build environment is sane... yes 2024-09-10T23:00:47.260 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-10T23:00:47.262 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gawk... gawk 2024-09-10T23:00:47.296 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T23:00:47.389 INFO:tasks.workunit.client.0.smithi067.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T23:00:47.389 INFO:tasks.workunit.client.0.smithi067.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T23:00:47.418 INFO:tasks.workunit.client.0.smithi067.stdout:checking for style of include used by make... GNU 2024-09-10T23:00:47.427 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... gcc 2024-09-10T23:00:47.580 INFO:tasks.workunit.client.0.smithi067.stdout:checking for C compiler default output file name... a.out 2024-09-10T23:00:47.586 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether the C compiler works... yes 2024-09-10T23:00:47.592 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are cross compiling... no 2024-09-10T23:00:47.680 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of executables... 2024-09-10T23:00:47.760 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of object files... o 2024-09-10T23:00:47.838 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T23:00:47.917 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... yes 2024-09-10T23:00:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:47 smithi067 ceph-mon[30939]: pgmap v444: 129 pgs: 129 active+clean; 3.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-09-10T23:00:48.065 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T23:00:48.257 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:47 smithi145 ceph-mon[34397]: pgmap v444: 129 pgs: 129 active+clean; 3.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-09-10T23:00:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:47 smithi092 ceph-mon[34223]: pgmap v444: 129 pgs: 129 active+clean; 3.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-09-10T23:00:48.311 INFO:tasks.workunit.client.0.smithi067.stdout:checking dependency style of gcc... gcc3 2024-09-10T23:00:48.463 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T23:00:48.577 INFO:tasks.workunit.client.0.smithi067.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T23:00:48.581 INFO:tasks.workunit.client.0.smithi067.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T23:00:48.608 INFO:tasks.workunit.client.0.smithi067.stdout:checking for AIX... no 2024-09-10T23:00:48.649 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... (cached) gcc 2024-09-10T23:00:48.677 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T23:00:48.678 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T23:00:48.679 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T23:00:48.680 INFO:tasks.workunit.client.0.smithi067.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T23:00:48.818 INFO:tasks.workunit.client.0.smithi067.stdout:checking for main in -lm... yes 2024-09-10T23:00:48.950 INFO:tasks.workunit.client.0.smithi067.stdout:checking for main in -lpthread... yes 2024-09-10T23:00:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ANSI C header files... yes 2024-09-10T23:00:49.466 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T23:00:49.569 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/types.h... yes 2024-09-10T23:00:49.679 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/stat.h... yes 2024-09-10T23:00:49.791 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdlib.h... yes 2024-09-10T23:00:49.913 INFO:tasks.workunit.client.0.smithi067.stdout:checking for string.h... yes 2024-09-10T23:00:50.031 INFO:tasks.workunit.client.0.smithi067.stdout:checking for memory.h... yes 2024-09-10T23:00:50.161 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strings.h... yes 2024-09-10T23:00:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:49 smithi092 ceph-mon[34223]: pgmap v445: 129 pgs: 129 active+clean; 3.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-10T23:00:50.276 INFO:tasks.workunit.client.0.smithi067.stdout:checking for inttypes.h... yes 2024-09-10T23:00:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:49 smithi067 ceph-mon[30939]: pgmap v445: 129 pgs: 129 active+clean; 3.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-10T23:00:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:49 smithi145 ceph-mon[34397]: pgmap v445: 129 pgs: 129 active+clean; 3.2 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-10T23:00:50.389 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdint.h... yes 2024-09-10T23:00:50.510 INFO:tasks.workunit.client.0.smithi067.stdout:checking for unistd.h... yes 2024-09-10T23:00:50.651 INFO:tasks.workunit.client.0.smithi067.stdout:checking pthread.h usability... yes 2024-09-10T23:00:50.705 INFO:tasks.workunit.client.0.smithi067.stdout:checking pthread.h presence... yes 2024-09-10T23:00:50.707 INFO:tasks.workunit.client.0.smithi067.stdout:checking for pthread.h... yes 2024-09-10T23:00:50.828 INFO:tasks.workunit.client.0.smithi067.stdout:checking fcntl.h usability... yes 2024-09-10T23:00:50.901 INFO:tasks.workunit.client.0.smithi067.stdout:checking fcntl.h presence... yes 2024-09-10T23:00:50.903 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fcntl.h... yes 2024-09-10T23:00:51.039 INFO:tasks.workunit.client.0.smithi067.stdout:checking limits.h usability... yes 2024-09-10T23:00:51.103 INFO:tasks.workunit.client.0.smithi067.stdout:checking limits.h presence... yes 2024-09-10T23:00:51.105 INFO:tasks.workunit.client.0.smithi067.stdout:checking for limits.h... yes 2024-09-10T23:00:51.148 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdint.h... (cached) yes 2024-09-10T23:00:51.262 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h usability... yes 2024-09-10T23:00:51.314 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h presence... yes 2024-09-10T23:00:51.315 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/time.h... yes 2024-09-10T23:00:51.349 INFO:tasks.workunit.client.0.smithi067.stdout:checking for unistd.h... (cached) yes 2024-09-10T23:00:51.466 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/vfs.h usability... yes 2024-09-10T23:00:51.521 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/vfs.h presence... yes 2024-09-10T23:00:51.523 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/vfs.h... yes 2024-09-10T23:00:51.634 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/limits.h usability... no 2024-09-10T23:00:51.693 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/limits.h presence... no 2024-09-10T23:00:51.694 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/limits.h... no 2024-09-10T23:00:51.775 INFO:tasks.workunit.client.0.smithi067.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T23:00:51.860 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T23:00:51.865 INFO:tasks.workunit.client.0.smithi067.stdout:checking for function prototypes... yes 2024-09-10T23:00:51.966 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether setvbuf arguments are reversed... no 2024-09-10T23:00:52.137 INFO:tasks.workunit.client.0.smithi067.stdout:checking for system... yes 2024-09-10T23:00:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:51 smithi092 ceph-mon[34223]: pgmap v446: 129 pgs: 129 active+clean; 3.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 240 KiB/s wr, 48 op/s 2024-09-10T23:00:52.297 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gettimeofday... yes 2024-09-10T23:00:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:51 smithi067 ceph-mon[30939]: pgmap v446: 129 pgs: 129 active+clean; 3.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 240 KiB/s wr, 48 op/s 2024-09-10T23:00:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:51 smithi145 ceph-mon[34397]: pgmap v446: 129 pgs: 129 active+clean; 3.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 240 KiB/s wr, 48 op/s 2024-09-10T23:00:52.452 INFO:tasks.workunit.client.0.smithi067.stdout:checking for mkdir... yes 2024-09-10T23:00:52.608 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strerror... yes 2024-09-10T23:00:52.760 INFO:tasks.workunit.client.0.smithi067.stdout:checking for open64... yes 2024-09-10T23:00:52.913 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stat64... yes 2024-09-10T23:00:53.069 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fseeko64... yes 2024-09-10T23:00:53.221 INFO:tasks.workunit.client.0.smithi067.stdout:checking for lrand48_r... yes 2024-09-10T23:00:53.378 INFO:tasks.workunit.client.0.smithi067.stdout:checking for srand48_r... yes 2024-09-10T23:00:53.426 INFO:tasks.workunit.client.0.smithi067.stdout:configure: creating ./config.status 2024-09-10T23:00:53.697 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating Makefile 2024-09-10T23:00:53.735 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating config.h 2024-09-10T23:00:53.785 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: executing depfiles commands 2024-09-10T23:00:53.931 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: executing default commands 2024-09-10T23:00:54.030 INFO:tasks.workunit.client.0.smithi067.stderr:+ make 2024-09-10T23:00:54.078 INFO:tasks.workunit.client.0.smithi067.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-10T23:00:54.104 INFO:tasks.workunit.client.0.smithi067.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-10T23:00:54.106 INFO:tasks.workunit.client.0.smithi067.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-10T23:00:54.106 INFO:tasks.workunit.client.0.smithi067.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-10T23:00:54.106 INFO:tasks.workunit.client.0.smithi067.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-10T23:00:54.106 INFO:tasks.workunit.client.0.smithi067.stderr: any GNU archive site. 2024-09-10T23:00:54.117 INFO:tasks.workunit.client.0.smithi067.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-10T23:00:54.127 INFO:tasks.workunit.client.0.smithi067.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-10T23:00:54.129 INFO:tasks.workunit.client.0.smithi067.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-10T23:00:54.129 INFO:tasks.workunit.client.0.smithi067.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-10T23:00:54.130 INFO:tasks.workunit.client.0.smithi067.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-10T23:00:54.130 INFO:tasks.workunit.client.0.smithi067.stderr: Grab them from any GNU archive site. 2024-09-10T23:00:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:53 smithi092 ceph-mon[34223]: pgmap v447: 129 pgs: 129 active+clean; 5.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 354 KiB/s wr, 56 op/s 2024-09-10T23:00:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:53 smithi067 ceph-mon[30939]: pgmap v447: 129 pgs: 129 active+clean; 5.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 354 KiB/s wr, 56 op/s 2024-09-10T23:00:54.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:53 smithi145 ceph-mon[34397]: pgmap v447: 129 pgs: 129 active+clean; 5.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 354 KiB/s wr, 56 op/s 2024-09-10T23:00:54.447 INFO:tasks.workunit.client.0.smithi067.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-10T23:00:54.879 INFO:tasks.workunit.client.0.smithi067.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-10T23:00:54.879 INFO:tasks.workunit.client.0.smithi067.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-10T23:00:54.880 INFO:tasks.workunit.client.0.smithi067.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-10T23:00:54.880 INFO:tasks.workunit.client.0.smithi067.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-10T23:00:55.036 INFO:tasks.workunit.client.0.smithi067.stdout:/bin/sh ./config.status --recheck 2024-09-10T23:00:55.179 INFO:tasks.workunit.client.0.smithi067.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-10T23:00:55.460 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T23:00:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:55 smithi092 ceph-mon[34223]: pgmap v448: 129 pgs: 129 active+clean; 5.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 379 KiB/s wr, 74 op/s 2024-09-10T23:00:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:55 smithi067 ceph-mon[30939]: pgmap v448: 129 pgs: 129 active+clean; 5.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 379 KiB/s wr, 74 op/s 2024-09-10T23:00:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:55 smithi145 ceph-mon[34397]: pgmap v448: 129 pgs: 129 active+clean; 5.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 379 KiB/s wr, 74 op/s 2024-09-10T23:00:56.477 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether build environment is sane... yes 2024-09-10T23:00:56.498 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-10T23:00:56.499 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gawk... gawk 2024-09-10T23:00:56.538 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T23:00:56.630 INFO:tasks.workunit.client.0.smithi067.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T23:00:56.630 INFO:tasks.workunit.client.0.smithi067.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T23:00:56.658 INFO:tasks.workunit.client.0.smithi067.stdout:checking for style of include used by make... GNU 2024-09-10T23:00:56.666 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... gcc 2024-09-10T23:00:56.865 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether the C compiler works... yes 2024-09-10T23:00:56.865 INFO:tasks.workunit.client.0.smithi067.stdout:checking for C compiler default output file name... a.out 2024-09-10T23:00:56.962 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of executables... 2024-09-10T23:00:57.058 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are cross compiling... no 2024-09-10T23:00:57.146 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of object files... o 2024-09-10T23:00:57.205 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T23:00:57.264 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... yes 2024-09-10T23:00:57.338 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T23:00:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:checking dependency style of gcc... gcc3 2024-09-10T23:00:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T23:00:57.791 INFO:tasks.workunit.client.0.smithi067.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T23:00:57.795 INFO:tasks.workunit.client.0.smithi067.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T23:00:58.116 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ANSI C header files... yes 2024-09-10T23:00:58.194 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/types.h... yes 2024-09-10T23:00:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:57 smithi092 ceph-mon[34223]: pgmap v449: 129 pgs: 129 active+clean; 6.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 490 KiB/s wr, 95 op/s 2024-09-10T23:00:58.284 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/stat.h... yes 2024-09-10T23:00:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:57 smithi067 ceph-mon[30939]: pgmap v449: 129 pgs: 129 active+clean; 6.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 490 KiB/s wr, 95 op/s 2024-09-10T23:00:58.375 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdlib.h... yes 2024-09-10T23:00:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:57 smithi145 ceph-mon[34397]: pgmap v449: 129 pgs: 129 active+clean; 6.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 490 KiB/s wr, 95 op/s 2024-09-10T23:00:58.469 INFO:tasks.workunit.client.0.smithi067.stdout:checking for string.h... yes 2024-09-10T23:00:58.564 INFO:tasks.workunit.client.0.smithi067.stdout:checking for memory.h... yes 2024-09-10T23:00:58.659 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strings.h... yes 2024-09-10T23:00:58.751 INFO:tasks.workunit.client.0.smithi067.stdout:checking for inttypes.h... yes 2024-09-10T23:00:58.842 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdint.h... yes 2024-09-10T23:00:58.940 INFO:tasks.workunit.client.0.smithi067.stdout:checking for unistd.h... yes 2024-09-10T23:00:59.037 INFO:tasks.workunit.client.0.smithi067.stdout:checking minix/config.h usability... no 2024-09-10T23:00:59.099 INFO:tasks.workunit.client.0.smithi067.stdout:checking minix/config.h presence... no 2024-09-10T23:00:59.100 INFO:tasks.workunit.client.0.smithi067.stdout:checking for minix/config.h... no 2024-09-10T23:00:59.175 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-10T23:00:59.188 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... (cached) gcc 2024-09-10T23:00:59.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:58 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:00:59.275 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T23:00:59.276 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T23:00:59.277 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T23:00:59.278 INFO:tasks.workunit.client.0.smithi067.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T23:00:59.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:58 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:00:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:58 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:00:59.387 INFO:tasks.workunit.client.0.smithi067.stdout:checking for main in -lm... yes 2024-09-10T23:00:59.498 INFO:tasks.workunit.client.0.smithi067.stdout:checking for main in -lpthread... yes 2024-09-10T23:00:59.502 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ANSI C header files... (cached) yes 2024-09-10T23:00:59.602 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T23:00:59.697 INFO:tasks.workunit.client.0.smithi067.stdout:checking pthread.h usability... yes 2024-09-10T23:00:59.738 INFO:tasks.workunit.client.0.smithi067.stdout:checking pthread.h presence... yes 2024-09-10T23:00:59.739 INFO:tasks.workunit.client.0.smithi067.stdout:checking for pthread.h... yes 2024-09-10T23:00:59.935 INFO:tasks.workunit.client.0.smithi067.stdout:checking fcntl.h usability... yes 2024-09-10T23:00:59.974 INFO:tasks.workunit.client.0.smithi067.stdout:checking fcntl.h presence... yes 2024-09-10T23:00:59.975 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fcntl.h... yes 2024-09-10T23:01:00.111 INFO:tasks.workunit.client.0.smithi067.stdout:checking limits.h usability... yes 2024-09-10T23:01:00.150 INFO:tasks.workunit.client.0.smithi067.stdout:checking limits.h presence... yes 2024-09-10T23:01:00.150 INFO:tasks.workunit.client.0.smithi067.stdout:checking for limits.h... yes 2024-09-10T23:01:00.184 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdint.h... (cached) yes 2024-09-10T23:01:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:00:59 smithi092 ceph-mon[34223]: pgmap v450: 129 pgs: 129 active+clean; 6.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 359 KiB/s wr, 80 op/s 2024-09-10T23:01:00.277 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h usability... yes 2024-09-10T23:01:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:00:59 smithi067 ceph-mon[30939]: pgmap v450: 129 pgs: 129 active+clean; 6.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 359 KiB/s wr, 80 op/s 2024-09-10T23:01:00.316 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h presence... yes 2024-09-10T23:01:00.316 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/time.h... yes 2024-09-10T23:01:00.345 INFO:tasks.workunit.client.0.smithi067.stdout:checking for unistd.h... (cached) yes 2024-09-10T23:01:00.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:00:59 smithi145 ceph-mon[34397]: pgmap v450: 129 pgs: 129 active+clean; 6.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 359 KiB/s wr, 80 op/s 2024-09-10T23:01:00.438 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/vfs.h usability... yes 2024-09-10T23:01:00.474 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/vfs.h presence... yes 2024-09-10T23:01:00.474 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/vfs.h... yes 2024-09-10T23:01:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:01 smithi067 ceph-mon[30939]: pgmap v451: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 499 KiB/s wr, 107 op/s 2024-09-10T23:01:01.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:01 smithi145 ceph-mon[34397]: pgmap v451: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 499 KiB/s wr, 107 op/s 2024-09-10T23:01:01.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:01 smithi092 ceph-mon[34223]: pgmap v451: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 499 KiB/s wr, 107 op/s 2024-09-10T23:01:02.584 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/limits.h usability... no 2024-09-10T23:01:02.670 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/limits.h presence... no 2024-09-10T23:01:02.671 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/limits.h... no 2024-09-10T23:01:02.738 INFO:tasks.workunit.client.0.smithi067.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T23:01:02.806 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T23:01:02.939 INFO:tasks.workunit.client.0.smithi067.stdout:checking for system... yes 2024-09-10T23:01:03.080 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gettimeofday... yes 2024-09-10T23:01:03.220 INFO:tasks.workunit.client.0.smithi067.stdout:checking for mkdir... yes 2024-09-10T23:01:03.358 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strerror... yes 2024-09-10T23:01:03.496 INFO:tasks.workunit.client.0.smithi067.stdout:checking for open64... yes 2024-09-10T23:01:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:03 smithi092 ceph-mon[34223]: pgmap v452: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 429 KiB/s wr, 91 op/s 2024-09-10T23:01:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:03 smithi067 ceph-mon[30939]: pgmap v452: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 429 KiB/s wr, 91 op/s 2024-09-10T23:01:03.629 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stat64... yes 2024-09-10T23:01:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:03 smithi145 ceph-mon[34397]: pgmap v452: 129 pgs: 129 active+clean; 7.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 429 KiB/s wr, 91 op/s 2024-09-10T23:01:03.767 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fseeko64... yes 2024-09-10T23:01:03.904 INFO:tasks.workunit.client.0.smithi067.stdout:checking for lrand48_r... yes 2024-09-10T23:01:04.040 INFO:tasks.workunit.client.0.smithi067.stdout:checking for srand48_r... yes 2024-09-10T23:01:04.088 INFO:tasks.workunit.client.0.smithi067.stdout:configure: creating ./config.status 2024-09-10T23:01:04.286 INFO:tasks.workunit.client.0.smithi067.stdout: /bin/sh ./config.status 2024-09-10T23:01:04.411 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating Makefile 2024-09-10T23:01:04.448 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating config.h 2024-09-10T23:01:04.497 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: executing depfiles commands 2024-09-10T23:01:04.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:04 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:01:04.541 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: executing default commands 2024-09-10T23:01:04.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:04 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:01:04.619 INFO:tasks.workunit.client.0.smithi067.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-10T23:01:04.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:04 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:01:04.891 INFO:tasks.workunit.client.0.smithi067.stdout:rm -f stamp-h1 2024-09-10T23:01:04.896 INFO:tasks.workunit.client.0.smithi067.stdout:touch config.h.in 2024-09-10T23:01:04.901 INFO:tasks.workunit.client.0.smithi067.stdout:cd . && /bin/sh ./config.status config.h 2024-09-10T23:01:05.007 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating config.h 2024-09-10T23:01:05.070 INFO:tasks.workunit.client.0.smithi067.stdout:make all-am 2024-09-10T23:01:05.128 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T23:01:05.128 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-10T23:01:05.397 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-10T23:01:05.406 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-10T23:01:05.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:05 smithi092 ceph-mon[34223]: pgmap v453: 129 pgs: 129 active+clean; 9.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 440 KiB/s wr, 100 op/s 2024-09-10T23:01:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:05 smithi067 ceph-mon[30939]: pgmap v453: 129 pgs: 129 active+clean; 9.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 440 KiB/s wr, 100 op/s 2024-09-10T23:01:05.563 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-10T23:01:05.571 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-10T23:01:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:05 smithi145 ceph-mon[34397]: pgmap v453: 129 pgs: 129 active+clean; 9.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 440 KiB/s wr, 100 op/s 2024-09-10T23:01:05.734 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-10T23:01:05.743 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-10T23:01:05.909 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-10T23:01:05.918 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-10T23:01:06.133 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-10T23:01:06.142 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-10T23:01:06.257 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-10T23:01:06.266 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-10T23:01:06.353 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-10T23:01:06.361 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-10T23:01:06.462 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-10T23:01:06.471 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-10T23:01:06.744 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-10T23:01:06.752 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-10T23:01:06.951 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-10T23:01:06.961 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-10T23:01:07.205 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-10T23:01:07.213 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-10T23:01:07.346 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-10T23:01:07.356 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-10T23:01:07.507 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-10T23:01:07.516 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-10T23:01:07.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:07 smithi092 ceph-mon[34223]: pgmap v454: 129 pgs: 129 active+clean; 9.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 433 KiB/s wr, 96 op/s 2024-09-10T23:01:07.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:07 smithi067 ceph-mon[30939]: pgmap v454: 129 pgs: 129 active+clean; 9.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 433 KiB/s wr, 96 op/s 2024-09-10T23:01:07.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:07 smithi145 ceph-mon[34397]: pgmap v454: 129 pgs: 129 active+clean; 9.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 433 KiB/s wr, 96 op/s 2024-09-10T23:01:07.735 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-10T23:01:07.753 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-10T23:01:07.834 INFO:tasks.workunit.client.0.smithi067.stderr:In file included from parser.h:22, 2024-09-10T23:01:07.834 INFO:tasks.workunit.client.0.smithi067.stderr: from parser.c:25: 2024-09-10T23:01:07.834 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c: In function ‘init_filesys’: 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.837 INFO:tasks.workunit.client.0.smithi067.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.838 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.839 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.840 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T23:01:07.841 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.842 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-10T23:01:07.842 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.842 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.842 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.842 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.843 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.844 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T23:01:07.845 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: 36 | container_of(ptr, type, member) 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~ 2024-09-10T23:01:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T23:01:07.847 INFO:tasks.workunit.client.0.smithi067.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T23:01:07.847 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c: In function ‘get_optstr’: 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:200:14: note: source object declared here 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr: 200 | char temp[BUFSIZE]; 2024-09-10T23:01:07.892 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~ 2024-09-10T23:01:07.900 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c: In function ‘size64_convert’: 2024-09-10T23:01:07.900 INFO:tasks.workunit.client.0.smithi067.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-10T23:01:07.900 INFO:tasks.workunit.client.0.smithi067.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-10T23:01:07.900 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-10T23:01:08.262 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-10T23:01:08.281 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-10T23:01:08.421 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-10T23:01:08.440 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-10T23:01:08.607 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-10T23:01:08.628 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-10T23:01:08.709 INFO:tasks.workunit.client.0.smithi067.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-10T23:01:08.731 INFO:tasks.workunit.client.0.smithi067.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-10T23:01:08.945 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T23:01:08.947 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .. 2024-09-10T23:01:08.949 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir tmp 2024-09-10T23:01:08.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd tmp 2024-09-10T23:01:08.957 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in $mydir/*.ffsb 2024-09-10T23:01:08.957 INFO:tasks.workunit.client.0.smithi067.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-10T23:01:08.959 INFO:tasks.workunit.client.0.smithi067.stdout:FFSB version 6.0-RC2 started 2024-09-10T23:01:08.959 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.966 INFO:tasks.workunit.client.0.smithi067.stdout:benchmark time = 300 2024-09-10T23:01:08.966 INFO:tasks.workunit.client.0.smithi067.stdout:ThreadGroup 0 2024-09-10T23:01:08.966 INFO:tasks.workunit.client.0.smithi067.stdout:================ 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: num_threads = 32 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: read_random = off 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: read_size = 0 (0B) 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: read_blocksize = 0 (0B) 2024-09-10T23:01:08.967 INFO:tasks.workunit.client.0.smithi067.stdout: read_skip = off 2024-09-10T23:01:08.968 INFO:tasks.workunit.client.0.smithi067.stdout: read_skipsize = 0 (0B) 2024-09-10T23:01:08.968 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.968 INFO:tasks.workunit.client.0.smithi067.stdout: write_random = on 2024-09-10T23:01:08.968 INFO:tasks.workunit.client.0.smithi067.stdout: write_size = 5242880 (5MB) 2024-09-10T23:01:08.968 INFO:tasks.workunit.client.0.smithi067.stdout: fsync_file = 0 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: write_blocksize = 4096 (4KB) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: wait time = 0 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: op weights 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: read = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: readall = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: write = 1 (100.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: create = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: append = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: delete = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: metaop = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: createdir = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: stat = 0 (0.00%) 2024-09-10T23:01:08.969 INFO:tasks.workunit.client.0.smithi067.stdout: writeall = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: writeall_fsync = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: open_close = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: write_fsync = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: create_fsync = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: append_fsync = 0 (0.00%) 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout:FileSystem . 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout:========== 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: num_dirs = 0 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: starting files = 128 2024-09-10T23:01:08.970 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: min file size = 104857600 (100MB) 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: max file size = 104857600 (100MB) 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: directio = off 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: alignedio = on 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: bufferedio = off 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: aging is off 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: current utilization = 0.04% 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout:checking existing fs: . 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stdout:recreating new fileset 2024-09-10T23:01:08.971 INFO:tasks.workunit.client.0.smithi067.stderr:opendir: No such file or directory 2024-09-10T23:01:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:09 smithi067 ceph-mon[30939]: pgmap v455: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 344 KiB/s wr, 91 op/s 2024-09-10T23:01:09.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:09 smithi092 ceph-mon[34223]: pgmap v455: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 344 KiB/s wr, 91 op/s 2024-09-10T23:01:09.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:09 smithi145 ceph-mon[34397]: pgmap v455: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 344 KiB/s wr, 91 op/s 2024-09-10T23:01:11.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:11 smithi092 ceph-mon[34223]: pgmap v456: 129 pgs: 129 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 476 KiB/s wr, 109 op/s 2024-09-10T23:01:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:11 smithi067 ceph-mon[30939]: pgmap v456: 129 pgs: 129 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 476 KiB/s wr, 109 op/s 2024-09-10T23:01:11.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:11 smithi145 ceph-mon[34397]: pgmap v456: 129 pgs: 129 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 476 KiB/s wr, 109 op/s 2024-09-10T23:01:13.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:13 smithi092 ceph-mon[34223]: pgmap v457: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 325 KiB/s wr, 81 op/s 2024-09-10T23:01:13.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:13 smithi067 ceph-mon[30939]: pgmap v457: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 325 KiB/s wr, 81 op/s 2024-09-10T23:01:13.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:13 smithi145 ceph-mon[34397]: pgmap v457: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 325 KiB/s wr, 81 op/s 2024-09-10T23:01:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:15 smithi092 ceph-mon[34223]: pgmap v458: 129 pgs: 129 active+clean; 39 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 81 op/s 2024-09-10T23:01:15.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:15 smithi067 ceph-mon[30939]: pgmap v458: 129 pgs: 129 active+clean; 39 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 81 op/s 2024-09-10T23:01:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:15 smithi145 ceph-mon[34397]: pgmap v458: 129 pgs: 129 active+clean; 39 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 81 op/s 2024-09-10T23:01:17.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:17 smithi092 ceph-mon[34223]: pgmap v459: 129 pgs: 129 active+clean; 79 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 67 op/s 2024-09-10T23:01:17.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:17 smithi067 ceph-mon[30939]: pgmap v459: 129 pgs: 129 active+clean; 79 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 67 op/s 2024-09-10T23:01:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:17 smithi145 ceph-mon[34397]: pgmap v459: 129 pgs: 129 active+clean; 79 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 67 op/s 2024-09-10T23:01:19.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:19 smithi092 ceph-mon[34223]: pgmap v460: 129 pgs: 129 active+clean; 91 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-09-10T23:01:19.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:19 smithi067 ceph-mon[30939]: pgmap v460: 129 pgs: 129 active+clean; 91 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-09-10T23:01:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:19 smithi145 ceph-mon[34397]: pgmap v460: 129 pgs: 129 active+clean; 91 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-09-10T23:01:20.196 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:01:20.196 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-10T23:01:20.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:21.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 -- 172.21.15.67:0/1128249413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0109130 msgr2=0x7efdb010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 --2- 172.21.15.67:0/1128249413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0109130 0x7efdb010f930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efda8009ca0 tx=0x7efda802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 -- 172.21.15.67:0/1128249413 shutdown_connections 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 --2- 172.21.15.67:0/1128249413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0109130 0x7efdb010f930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 --2- 172.21.15.67:0/1128249413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 0x7efdb0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 --2- 172.21.15.67:0/1128249413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0102da0 0x7efdb0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.244+0000 7efdb5a00640 1 -- 172.21.15.67:0/1128249413 >> 172.21.15.67:0/1128249413 conn(0x7efdb00fe810 msgr2=0x7efdb0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.245+0000 7efdb5a00640 1 -- 172.21.15.67:0/1128249413 shutdown_connections 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.245+0000 7efdb5a00640 1 -- 172.21.15.67:0/1128249413 wait complete. 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 Processor -- start 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 -- start start 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 0x7efdb019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0109130 0x7efdb01a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0112cb0 con 0x7efdb0102da0 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0112b30 con 0x7efdb01087b0 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.246+0000 7efdb5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0112e30 con 0x7efdb0109130 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50574/0 (socket says 172.21.15.67:50574) 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 -- 172.21.15.67:0/3013677461 learned_addr learned my addr 172.21.15.67:0/3013677461 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaf400640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0109130 0x7efdb01a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efda7e00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 0x7efdb019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 -- 172.21.15.67:0/3013677461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0109130 msgr2=0x7efdb01a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0109130 0x7efdb01a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 -- 172.21.15.67:0/3013677461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 msgr2=0x7efdb019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 0x7efdb019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.247+0000 7efdaea00640 1 -- 172.21.15.67:0/3013677461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb01a1430 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efdaea00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efda000d950 tx=0x7efda000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0017070 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda0009c30 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda000f620 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb01a1720 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.248+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb01a7fb0 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.250+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb0104050 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.252+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efda0010460 con 0x7efdb0102da0 2024-09-10T23:01:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.254+0000 7efda7400640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 0x7efd940b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.254+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7efda00e5ee0 con 0x7efdb0102da0 2024-09-10T23:01:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.254+0000 7efda7e00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 0x7efd940b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.255+0000 7efda7e00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 0x7efd940b7990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efdb0104f60 tx=0x7efd98005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.258+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efda0009de0 con 0x7efdb0102da0 2024-09-10T23:01:21.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.515+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdb01a81f0 con 0x7efdb0102da0 2024-09-10T23:01:21.517 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:21 smithi067 ceph-mon[30939]: pgmap v461: 129 pgs: 129 active+clean; 155 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 40 op/s 2024-09-10T23:01:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.516+0000 7efda7400640 1 -- 172.21.15.67:0/3013677461 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7efda00ac020 con 0x7efdb0102da0 2024-09-10T23:01:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:01:21.518 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-09-10T23:01:03:188481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:01:02.557752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:01:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-09-10T23:01:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.518+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 msgr2=0x7efd940b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.518+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 0x7efd940b7990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efdb0104f60 tx=0x7efd98005f90 comp rx=0 tx=0).stop 2024-09-10T23:01:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.518+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 msgr2=0x7efdb019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.518+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efda000d950 tx=0x7efda000de10 comp rx=0 tx=0).stop 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 shutdown_connections 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b54d0 0x7efd940b7990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb0109130 0x7efdb01a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb01087b0 0x7efdb019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 --2- 172.21.15.67:0/3013677461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0102da0 0x7efdb019b380 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 >> 172.21.15.67:0/3013677461 conn(0x7efdb00fe810 msgr2=0x7efdb0075430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 shutdown_connections 2024-09-10T23:01:21.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:21 smithi092 ceph-mon[34223]: pgmap v461: 129 pgs: 129 active+clean; 155 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 40 op/s 2024-09-10T23:01:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:21.519+0000 7efdb5a00640 1 -- 172.21.15.67:0/3013677461 wait complete. 2024-09-10T23:01:21.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:21 smithi145 ceph-mon[34397]: pgmap v461: 129 pgs: 129 active+clean; 155 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 40 op/s 2024-09-10T23:01:22.426 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-10T23:01:22.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-10T23:01:22.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:22 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3013677461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:22.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:22 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3013677461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:22.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:22 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3013677461' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:23.501 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:23 smithi067 ceph-mon[30939]: pgmap v462: 129 pgs: 129 active+clean; 167 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 -- 172.21.15.67:0/395240173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78102c10 msgr2=0x7f7b78102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 --2- 172.21.15.67:0/395240173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78102c10 0x7f7b78102ff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009900 tx=0x7f7b6802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 -- 172.21.15.67:0/395240173 shutdown_connections 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 --2- 172.21.15.67:0/395240173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108fa0 0x7f7b7810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 --2- 172.21.15.67:0/395240173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78108620 0x7f7b78108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 --2- 172.21.15.67:0/395240173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78102c10 0x7f7b78102ff0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 -- 172.21.15.67:0/395240173 >> 172.21.15.67:0/395240173 conn(0x7f7b780fe660 msgr2=0x7f7b78100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 -- 172.21.15.67:0/395240173 shutdown_connections 2024-09-10T23:01:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.505+0000 7f7b77200640 1 -- 172.21.15.67:0/395240173 wait complete. 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.506+0000 7f7b77200640 1 Processor -- start 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.506+0000 7f7b77200640 1 -- start start 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 0x7f7b78198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 0x7f7b78199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78112b60 con 0x7f7b78108fa0 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781129e0 con 0x7f7b78102c10 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b77200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78112ce0 con 0x7f7b78108620 2024-09-10T23:01:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b76800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b75e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 0x7f7b78198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b76800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50592/0 (socket says 172.21.15.67:50592) 2024-09-10T23:01:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b75e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 0x7f7b78198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:50334/0 (socket says 172.21.15.67:50334) 2024-09-10T23:01:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b76800640 1 -- 172.21.15.67:0/1043803889 learned_addr learned my addr 172.21.15.67:0/1043803889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.507+0000 7f7b75400640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 0x7f7b78199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b75400640 1 -- 172.21.15.67:0/1043803889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 msgr2=0x7f7b78198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b75400640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 0x7f7b78198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b75400640 1 -- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 msgr2=0x7f7b7819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b75400640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b75400640 1 -- 172.21.15.67:0/1043803889 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7819f080 con 0x7f7b78108620 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.508+0000 7f7b76800640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b75400640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 0x7f7b78199510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9f0 tx=0x7f7b6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b6ea00640 1 -- 172.21.15.67:0/1043803889 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400ccc0 con 0x7f7b78108620 2024-09-10T23:01:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b6ea00640 1 -- 172.21.15.67:0/1043803889 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b640108c0 con 0x7f7b78108620 2024-09-10T23:01:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7819f370 con 0x7f7b78108620 2024-09-10T23:01:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b6ea00640 1 -- 172.21.15.67:0/1043803889 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640187e0 con 0x7f7b78108620 2024-09-10T23:01:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.509+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b78075550 con 0x7f7b78108620 2024-09-10T23:01:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 Processor -- start 2024-09-10T23:01:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 -- start start 2024-09-10T23:01:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7830ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564ef779dca0 con 0x564ef7831420 2024-09-10T23:01:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ef778d0d0 con 0x564ef783d3b0 2024-09-10T23:01:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.514+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564ef7790ee0 con 0x564ef7830b00 2024-09-10T23:01:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.515+0000 7f7b76800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7830ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.515+0000 7f7b75e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.515+0000 7f7b76800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7830ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:60642/0 (socket says 172.21.15.67:60642) 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.515+0000 7f7b75e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50602/0 (socket says 172.21.15.67:50602) 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.515+0000 7f7b76800640 1 -- 172.21.15.67:0/3348390063 learned_addr learned my addr 172.21.15.67:0/3348390063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b6ea00640 1 -- 172.21.15.67:0/1043803889 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7b64010430 con 0x7f7b78108620 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 -- 172.21.15.67:0/3348390063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 msgr2=0x564ef7830ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7830ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 -- 172.21.15.67:0/3348390063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 msgr2=0x564ef783f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 -- 172.21.15.67:0/3348390063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef773f400 con 0x564ef7831420 2024-09-10T23:01:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.516+0000 7f7b75e00640 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802a910 tx=0x7f7b68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff308769996c06b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.517+0000 7f7b63400640 1 -- 172.21.15.67:0/3348390063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68040070 con 0x564ef7831420 2024-09-10T23:01:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.517+0000 7f7b63400640 1 -- 172.21.15.67:0/3348390063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b680054b0 con 0x564ef7831420 2024-09-10T23:01:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.517+0000 7f7b63400640 1 -- 172.21.15.67:0/3348390063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68035040 con 0x564ef7831420 2024-09-10T23:01:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.518+0000 7f7b6ea00640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 0x7f7b5c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.518+0000 7f7b6ea00640 1 -- 172.21.15.67:0/1043803889 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f7b640e6680 con 0x7f7b78108620 2024-09-10T23:01:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.518+0000 7f7b76800640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 0x7f7b5c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.518+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/3348390063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 msgr2=0x564ef783ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.518+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802a910 tx=0x7f7b68000bc0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/3348390063 shutdown_connections 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783ce70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/3348390063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7830ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/3348390063 >> 172.21.15.67:0/3348390063 conn(0x564ef782e930 msgr2=0x564ef782fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.522 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:23 smithi092 ceph-mon[34223]: pgmap v462: 129 pgs: 129 active+clean; 167 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-09-10T23:01:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/3348390063 shutdown_connections 2024-09-10T23:01:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.519+0000 7f7b76800640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 0x7f7b5c0b7ac0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009370 tx=0x7f7b70009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.520+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/3348390063 wait complete. 2024-09-10T23:01:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.520+0000 7f7b7fc3f740 1 -- start start 2024-09-10T23:01:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 Processor -- start 2024-09-10T23:01:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef783ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564ef75ee7a0 con 0x564ef7831420 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ef7732a10 con 0x564ef783d3b0 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564ef783ffd0 con 0x564ef7830b00 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.521+0000 7f7b7fc3f740 10 client.0 _ll_register_callbacks cb 0x564ef7742be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:50362/0 (socket says 172.21.15.67:50362) 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 -- 172.21.15.67:0/211581589 learned_addr learned my addr 172.21.15.67:0/211581589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b75e00640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef783ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 -- 172.21.15.67:0/211581589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 msgr2=0x564ef783ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef783ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 -- 172.21.15.67:0/211581589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 msgr2=0x564ef783a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.522+0000 7f7b76800640 1 -- 172.21.15.67:0/211581589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef74fab90 con 0x564ef783d3b0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.523+0000 7f7b76800640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b70011c50 tx=0x7f7b70004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399be79b6e9643d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.523+0000 7f7b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.523+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7002c060 con 0x564ef783d3b0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.523+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f7b70017030 con 0x564ef783d3b0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.523+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b680054b0 con 0x564ef783d3b0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.524+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b70011e90 con 0x564ef783d3b0 2024-09-10T23:01:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.524+0000 7f7b7fc3f740 10 client.35462 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.524+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ef784b980 con 0x564ef783d3b0 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.524+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b680054b0 con 0x564ef783d3b0 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.524+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b7001c430 con 0x564ef783d3b0 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 10 client.35462 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 10 client.35462 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 20 client.35462 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef784ce30 0x564ef784f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 4 client.35462 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef779fbf0 con 0x564ef784ce30 2024-09-10T23:01:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.525+0000 7f7b75e00640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef784ce30 0x564ef784f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.526+0000 7f7b75e00640 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef784ce30 0x564ef784f2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeedf5bf558dcf13 server_cookie=45cb3c88acbaa725 in_seq=0 out_seq=0 2024-09-10T23:01:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.526+0000 7f7b57e00640 10 client.35462 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:23.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b57e00640 1 -- 172.21.15.67:0/211581589 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ef779fbf0 con 0x564ef784ce30 2024-09-10T23:01:23.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b57e00640 10 client.35462 handle_command_reply: tid=0 2024-09-10T23:01:23.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 client.35462 shutdown 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 msgr2=0x564ef783a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b70011c50 tx=0x7f7b70004ab0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 shutdown_connections 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef784ce30 0x564ef784f2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef783a710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef783a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/211581589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef783ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 >> 172.21.15.67:0/211581589 conn(0x564ef782e930 msgr2=0x564ef7825300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.527+0000 7f7b57e00640 0 client.35462 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.528+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 shutdown_connections 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.528+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/211581589 wait complete. 2024-09-10T23:01:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.528+0000 7f7b7fc3f740 20 client.35462 trim_cache size 0 max 16384 2024-09-10T23:01:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 Processor -- start 2024-09-10T23:01:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 -- start start 2024-09-10T23:01:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7830b00 0x564ef7839990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 0x564ef7837850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781b0280 con 0x564ef7830b00 2024-09-10T23:01:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781adb50 con 0x564ef7831420 2024-09-10T23:01:23.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.552+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781ab9a0 con 0x564ef783d3b0 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.553+0000 7f7b75e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7830b00 0x564ef7839990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.553+0000 7f7b75400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.553+0000 7f7b75400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:50372/0 (socket says 172.21.15.67:50372) 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.553+0000 7f7b75400640 1 -- 172.21.15.67:0/366852231 learned_addr learned my addr 172.21.15.67:0/366852231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.553+0000 7f7b76800640 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 0x564ef7837850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 -- 172.21.15.67:0/366852231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 msgr2=0x564ef7837850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 0x564ef7837850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 -- 172.21.15.67:0/366852231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7830b00 msgr2=0x564ef7839990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7830b00 0x564ef7839990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 -- 172.21.15.67:0/366852231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef784f970 con 0x564ef7831420 2024-09-10T23:01:23.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b76800640 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 0x564ef7837850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:23.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.554+0000 7f7b75400640 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b640a9a30 tx=0x7f7b640a8770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11aac405fdac0f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.555+0000 7f7b62a00640 1 -- 172.21.15.67:0/366852231 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640b11f0 con 0x564ef7831420 2024-09-10T23:01:23.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.555+0000 7f7b62a00640 1 -- 172.21.15.67:0/366852231 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b640b5ca0 con 0x564ef7831420 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.555+0000 7f7b62a00640 1 -- 172.21.15.67:0/366852231 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640a8110 con 0x564ef7831420 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.555+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/366852231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 msgr2=0x564ef7837310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.555+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b640a9a30 tx=0x7f7b640a8770 comp rx=0 tx=0).stop 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/366852231 shutdown_connections 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef783d3b0 0x564ef7837850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef7831420 0x564ef7837310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/366852231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7830b00 0x564ef7839990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/366852231 >> 172.21.15.67:0/366852231 conn(0x564ef782e930 msgr2=0x564ef7825300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/366852231 shutdown_connections 2024-09-10T23:01:23.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/366852231 wait complete. 2024-09-10T23:01:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.556+0000 7f7b7fc3f740 1 -- start start 2024-09-10T23:01:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 Processor -- start 2024-09-10T23:01:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7845c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef78409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781ac010 con 0x564ef7831420 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781ad480 con 0x564ef783d3b0 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781ac6e0 con 0x564ef7830b00 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b7fc3f740 10 client.0 _ll_register_callbacks cb 0x7f7b70010f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.557+0000 7f7b75400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50620/0 (socket says 172.21.15.67:50620) 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 -- 172.21.15.67:0/2963210828 learned_addr learned my addr 172.21.15.67:0/2963210828 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75e00640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7845c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b76800640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef78409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 -- 172.21.15.67:0/2963210828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 msgr2=0x564ef7845c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7845c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 -- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 msgr2=0x564ef78409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef78409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b75400640 1 -- 172.21.15.67:0/2963210828 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef78b9dc0 con 0x564ef7831420 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.558+0000 7f7b76800640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef78409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b75400640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9c0 tx=0x7f7b64009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5bfb26901c9f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640e3e70 con 0x564ef7831420 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f7b640abc20 con 0x564ef7831420 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b640b5ca0 con 0x564ef7831420 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b640b11f0 con 0x564ef7831420 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b7fc3f740 10 client.15717 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:23.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.559+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ef78bb050 con 0x564ef7831420 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.560+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b640b5ca0 con 0x564ef7831420 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.560+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b640b2580 con 0x564ef7831420 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 10 client.15717 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 10 client.15717 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 20 client.15717 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef78bb790 0x564ef78bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 4 client.15717 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x564ef779b6a0 con 0x564ef78bb790 2024-09-10T23:01:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.561+0000 7f7b75e00640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef78bb790 0x564ef78bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.562+0000 7f7b75e00640 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef78bb790 0x564ef78bdbb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6860cc17b4a108f server_cookie=61b1739b8b75a9e7 in_seq=0 out_seq=0 2024-09-10T23:01:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.562+0000 7f7b57e00640 10 client.15717 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:23.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:23 smithi145 ceph-mon[34397]: pgmap v462: 129 pgs: 129 active+clean; 167 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-09-10T23:01:23.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b57e00640 1 -- 172.21.15.67:0/2963210828 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564ef779b6a0 con 0x564ef78bb790 2024-09-10T23:01:23.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b57e00640 10 client.15717 handle_command_reply: tid=0 2024-09-10T23:01:23.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 client.15717 shutdown 2024-09-10T23:01:23.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 msgr2=0x564ef7840220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9c0 tx=0x7f7b64009700 comp rx=0 tx=0).stop 2024-09-10T23:01:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 shutdown_connections 2024-09-10T23:01:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564ef78bb790 0x564ef78bdbb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ef783d3b0 0x564ef78409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ef7831420 0x564ef7840220 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 --2- 172.21.15.67:0/2963210828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ef7830b00 0x564ef7845c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 >> 172.21.15.67:0/2963210828 conn(0x564ef782e930 msgr2=0x564ef783b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b57e00640 0 client.15717 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.701+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 shutdown_connections 2024-09-10T23:01:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b7fc3f740 1 -- 172.21.15.67:0/2963210828 wait complete. 2024-09-10T23:01:23.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b7fc3f740 20 client.15717 trim_cache size 0 max 16384 2024-09-10T23:01:23.707 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:01:23.708 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 29, 2024-09-10T23:01:23.708 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:01:23.708 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 6, 2024-09-10T23:01:23.708 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "2ec564e5": { 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 6, 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:01:23.709 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 140.5, 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:01:23.710 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:23.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:23.712 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 msgr2=0x7f7b5c0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 0x7f7b5c0b7ac0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009370 tx=0x7f7b70009760 comp rx=0 tx=0).stop 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 msgr2=0x7f7b78199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.702+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 0x7f7b78199510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9f0 tx=0x7f7b6400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 shutdown_connections 2024-09-10T23:01:23.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7b5c0b5600 0x7f7b5c0b7ac0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b78108fa0 0x7f7b7819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b78108620 0x7f7b78199510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 --2- 172.21.15.67:0/1043803889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b78102c10 0x7f7b78198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 >> 172.21.15.67:0/1043803889 conn(0x7f7b780fe660 msgr2=0x7f7b780ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 shutdown_connections 2024-09-10T23:01:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:23.703+0000 7f7b77200640 1 -- 172.21.15.67:0/1043803889 wait complete. 2024-09-10T23:01:24.593 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2ec564e5', quiesce took 4.4 seconds. Will release after: 16.1 2024-09-10T23:01:25.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:25 smithi092 ceph-mon[34223]: pgmap v463: 129 pgs: 129 active+clean; 199 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T23:01:25.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:25 smithi067 ceph-mon[30939]: pgmap v463: 129 pgs: 129 active+clean; 199 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T23:01:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:25 smithi145 ceph-mon[34397]: pgmap v463: 129 pgs: 129 active+clean; 199 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T23:01:27.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:27 smithi092 ceph-mon[34223]: pgmap v464: 129 pgs: 129 active+clean; 227 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:01:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:27 smithi067 ceph-mon[30939]: pgmap v464: 129 pgs: 129 active+clean; 227 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:01:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:27 smithi145 ceph-mon[34397]: pgmap v464: 129 pgs: 129 active+clean; 227 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:01:29.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:29 smithi092 ceph-mon[34223]: pgmap v465: 129 pgs: 129 active+clean; 227 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-10T23:01:29.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:29 smithi067 ceph-mon[30939]: pgmap v465: 129 pgs: 129 active+clean; 227 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-10T23:01:29.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:29 smithi145 ceph-mon[34397]: pgmap v465: 129 pgs: 129 active+clean; 227 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-09-10T23:01:31.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:31 smithi092 ceph-mon[34223]: pgmap v466: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-09-10T23:01:31.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:31 smithi067 ceph-mon[30939]: pgmap v466: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-09-10T23:01:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:31 smithi145 ceph-mon[34397]: pgmap v466: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-09-10T23:01:33.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:33 smithi092 ceph-mon[34223]: pgmap v467: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-10T23:01:33.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:33 smithi067 ceph-mon[30939]: pgmap v467: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-10T23:01:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:33 smithi145 ceph-mon[34397]: pgmap v467: 129 pgs: 129 active+clean; 232 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-10T23:01:35.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:35 smithi067 ceph-mon[30939]: pgmap v468: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 11 op/s 2024-09-10T23:01:35.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:35 smithi145 ceph-mon[34397]: pgmap v468: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 11 op/s 2024-09-10T23:01:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:35 smithi092 ceph-mon[34223]: pgmap v468: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 11 op/s 2024-09-10T23:01:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:37 smithi067 ceph-mon[30939]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-09-10T23:01:37.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:37 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:37 smithi145 ceph-mon[34397]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-09-10T23:01:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:37 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:37 smithi092 ceph-mon[34223]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-09-10T23:01:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:37 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:38.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:38.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:38.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:01:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:38.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:01:38.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:38.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:38.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:01:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:39 smithi145 ceph-mon[34397]: pgmap v470: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 473 KiB/s wr, 6 op/s 2024-09-10T23:01:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:39 smithi092 ceph-mon[34223]: pgmap v470: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 473 KiB/s wr, 6 op/s 2024-09-10T23:01:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:39 smithi067 ceph-mon[30939]: pgmap v470: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 473 KiB/s wr, 6 op/s 2024-09-10T23:01:40.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:41.090 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.669+0000 7fbac2400640 1 -- 172.21.15.67:0/3808195310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc075e60 msgr2=0x7fbabc1135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.669+0000 7fbac2400640 1 --2- 172.21.15.67:0/3808195310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc075e60 0x7fbabc1135e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009900 tx=0x7fbaa802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 -- 172.21.15.67:0/3808195310 shutdown_connections 2024-09-10T23:01:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 --2- 172.21.15.67:0/3808195310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc113b20 0x7fbabc115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 --2- 172.21.15.67:0/3808195310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc075e60 0x7fbabc1135e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 --2- 172.21.15.67:0/3808195310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 0x7fbabc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 -- 172.21.15.67:0/3808195310 >> 172.21.15.67:0/3808195310 conn(0x7fbabc0fe830 msgr2=0x7fbabc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 -- 172.21.15.67:0/3808195310 shutdown_connections 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 -- 172.21.15.67:0/3808195310 wait complete. 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.670+0000 7fbac2400640 1 Processor -- start 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 -- start start 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 0x7fbabc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc075e60 0x7fbabc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc1192a0 con 0x7fbabc113b20 2024-09-10T23:01:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc119120 con 0x7fbabc075540 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbac2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc119420 con 0x7fbabc075e60 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbabbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbabbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49044/0 (socket says 172.21.15.67:49044) 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbabbe00640 1 -- 172.21.15.67:0/4042650641 learned_addr learned my addr 172.21.15.67:0/4042650641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbabb400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 0x7fbabc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.671+0000 7fbabaa00640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc075e60 0x7fbabc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 -- 172.21.15.67:0/4042650641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc075e60 msgr2=0x7fbabc19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc075e60 0x7fbabc19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 -- 172.21.15.67:0/4042650641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 msgr2=0x7fbabc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 0x7fbabc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 -- 172.21.15.67:0/4042650641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc1a5840 con 0x7fbabc113b20 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbabbe00640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbab400dac0 tx=0x7fbab400dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4017070 con 0x7fbabc113b20 2024-09-10T23:01:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbab4003e00 con 0x7fbabc113b20 2024-09-10T23:01:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc1a5b30 con 0x7fbabc113b20 2024-09-10T23:01:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.672+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4010460 con 0x7fbabc113b20 2024-09-10T23:01:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.673+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc1ac1f0 con 0x7fbabc113b20 2024-09-10T23:01:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.674+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc076a50 con 0x7fbabc113b20 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.676+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbab4004020 con 0x7fbabc113b20 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.677+0000 7fbab3e00640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 0x7fbaa00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.677+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fbab40e5fc0 con 0x7fbabc113b20 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.678+0000 7fbabb400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 0x7fbaa00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.678+0000 7fbabb400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 0x7fbaa00b7910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00abe0 tx=0x7fbaac00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.679+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbab40ac750 con 0x7fbabc113b20 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.915+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbabc075920 con 0x7fbabc113b20 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.915+0000 7fbab3e00640 1 -- 172.21.15.67:0/4042650641 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7fbab40ac050 con 0x7fbabc113b20 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-09-10T23:01:03:188481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:01:02.557752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-09-10T23:01:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 msgr2=0x7fbaa00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 0x7fbaa00b7910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00abe0 tx=0x7fbaac00a2d0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 msgr2=0x7fbabc1a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbab400dac0 tx=0x7fbab400dec0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 shutdown_connections 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbaa00b5450 0x7fbaa00b7910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbabc113b20 0x7fbabc1a5070 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc075e60 0x7fbabc19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 --2- 172.21.15.67:0/4042650641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbabc075540 0x7fbabc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 >> 172.21.15.67:0/4042650641 conn(0x7fbabc0fe830 msgr2=0x7fbabc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 shutdown_connections 2024-09-10T23:01:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:41.918+0000 7fbac2400640 1 -- 172.21.15.67:0/4042650641 wait complete. 2024-09-10T23:01:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:41 smithi145 ceph-mon[34397]: pgmap v471: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 474 KiB/s wr, 8 op/s 2024-09-10T23:01:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:41 smithi092 ceph-mon[34223]: pgmap v471: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 474 KiB/s wr, 8 op/s 2024-09-10T23:01:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:41 smithi067 ceph-mon[30939]: pgmap v471: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 474 KiB/s wr, 8 op/s 2024-09-10T23:01:42.886 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 2ec564e5 --release --await' 2024-09-10T23:01:42.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 2ec564e5 --release --await 2024-09-10T23:01:43.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:42 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4042650641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:43.268 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:43.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:42 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4042650641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:43.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:42 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4042650641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.865+0000 7f37bbe00640 1 -- 172.21.15.67:0/855810585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 msgr2=0x7f37bc111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.865+0000 7f37bbe00640 1 --2- 172.21.15.67:0/855810585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc111980 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 -- 172.21.15.67:0/855810585 shutdown_connections 2024-09-10T23:01:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 --2- 172.21.15.67:0/855810585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc111980 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 --2- 172.21.15.67:0/855810585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 0x7f37bc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 --2- 172.21.15.67:0/855810585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 0x7f37bc1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 -- 172.21.15.67:0/855810585 >> 172.21.15.67:0/855810585 conn(0x7f37bc100840 msgr2=0x7f37bc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 -- 172.21.15.67:0/855810585 shutdown_connections 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.866+0000 7f37bbe00640 1 -- 172.21.15.67:0/855810585 wait complete. 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 Processor -- start 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 -- start start 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 0x7f37bc19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 0x7f37bc19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc114d00 con 0x7f37bc10b180 2024-09-10T23:01:43.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc114b80 con 0x7f37bc10a800 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc114e80 con 0x7f37bc104df0 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49066/0 (socket says 172.21.15.67:49066) 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37bb400640 1 -- 172.21.15.67:0/554271580 learned_addr learned my addr 172.21.15.67:0/554271580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37baa00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 0x7f37bc19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.867+0000 7f37ba000640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 0x7f37bc19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 -- 172.21.15.67:0/554271580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 msgr2=0x7f37bc19b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 0x7f37bc19b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 -- 172.21.15.67:0/554271580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 msgr2=0x7f37bc19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 0x7f37bc19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 -- 172.21.15.67:0/554271580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc1a1200 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.868+0000 7f37bb400640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37b4031040 tx=0x7f37b4032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.869+0000 7f37c0a00640 1 -- 172.21.15.67:0/554271580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4004040 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.869+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1a1490 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.869+0000 7f37c0a00640 1 -- 172.21.15.67:0/554271580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b4004670 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.869+0000 7f37c0a00640 1 -- 172.21.15.67:0/554271580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4030bb0 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.869+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a7bb0 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.872+0000 7f37c48bb740 1 Processor -- start 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.872+0000 7f37c0a00640 1 -- 172.21.15.67:0/554271580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f37b40041e0 con 0x7f37bc10b180 2024-09-10T23:01:43.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.872+0000 7f37c48bb740 1 -- start start 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56dddbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a56dd44290 con 0x55a56ddcd1d0 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a56dd336c0 con 0x55a56ddd9a80 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37c48bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a56dd374d0 con 0x55a56ddcdaf0 2024-09-10T23:01:43.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37ba000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37baa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37baa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:48494/0 (socket says 172.21.15.67:48494) 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.873+0000 7f37baa00640 1 -- 172.21.15.67:0/769672511 learned_addr learned my addr 172.21.15.67:0/769672511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37ba000640 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56dddbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 -- 172.21.15.67:0/769672511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 msgr2=0x55a56dddbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56dddbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 -- 172.21.15.67:0/769672511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 msgr2=0x55a56ddcd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 -- 172.21.15.67:0/769672511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a56dcc0460 con 0x55a56ddcdaf0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37ba000640 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56dddbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37c0a00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 0x7f37a00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37baa00640 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b790 tx=0x7f37ac00bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f79288ab8c8a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37c0a00640 1 -- 172.21.15.67:0/554271580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f37b4107510 con 0x7f37bc10b180 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37ba000640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 0x7f37a00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37a7e00640 1 -- 172.21.15.67:0/769672511 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003dd0 con 0x55a56ddcdaf0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37a7e00640 1 -- 172.21.15.67:0/769672511 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37ac015070 con 0x55a56ddcdaf0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.874+0000 7f37a7e00640 1 -- 172.21.15.67:0/769672511 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004d60 con 0x55a56ddcdaf0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 -- 172.21.15.67:0/769672511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 msgr2=0x55a56ddd9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b790 tx=0x7f37ac00bc50 comp rx=0 tx=0).stop 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37ba000640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 0x7f37a00b7a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a56ddcd180 tx=0x7f37a800b650 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 -- 172.21.15.67:0/769672511 shutdown_connections 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56dddbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd9540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 --2- 172.21.15.67:0/769672511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 -- 172.21.15.67:0/769672511 >> 172.21.15.67:0/769672511 conn(0x55a56ddcb000 msgr2=0x55a56ddcc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 -- 172.21.15.67:0/769672511 shutdown_connections 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.875+0000 7f37c48bb740 1 -- 172.21.15.67:0/769672511 wait complete. 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.876+0000 7f37c48bb740 1 -- start start 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.876+0000 7f37c48bb740 1 Processor -- start 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56ddd74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a56db94bb0 con 0x55a56ddcd1d0 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a56dcce6c0 con 0x55a56ddd9a80 2024-09-10T23:01:43.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a56dddc730 con 0x55a56ddcdaf0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37c48bb740 10 client.0 _ll_register_callbacks cb 0x55a56dce9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49090/0 (socket says 172.21.15.67:49090) 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 -- 172.21.15.67:0/1716388589 learned_addr learned my addr 172.21.15.67:0/1716388589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37ba000640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 -- 172.21.15.67:0/1716388589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 msgr2=0x55a56ddd6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37bb400640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56ddd74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 -- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 msgr2=0x55a56ddd74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56ddd74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37baa00640 1 -- 172.21.15.67:0/1716388589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a56da91b00 con 0x55a56ddcd1d0 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37ba000640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:43.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.877+0000 7f37bb400640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56ddd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37baa00640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539051198138da2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac011640 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac015070 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f37ac0117e0 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37ac0119e0 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37c48bb740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.878+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a56db08960 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37ac015070 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37ac019c20 con 0x55a56ddcd1d0 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 10 client.15735 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:43.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 20 client.15735 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:43.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56dde94d0 0x55a56ddeb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 4 client.15735 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:43.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a56dd461e0 con 0x55a56dde94d0 2024-09-10T23:01:43.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.879+0000 7f37ba000640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56dde94d0 0x55a56ddeb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.880+0000 7f37ba000640 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56dde94d0 0x55a56ddeb8c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed23893bbbae7b5 server_cookie=430cdfa2f82a597e in_seq=0 out_seq=0 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.880+0000 7f37a4c00640 10 client.15735 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37a4c00640 1 -- 172.21.15.67:0/1716388589 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a56dd461e0 con 0x55a56dde94d0 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37a4c00640 10 client.15735 handle_command_reply: tid=0 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 client.15735 shutdown 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 msgr2=0x55a56ddd6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac003bd0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 shutdown_connections 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56dde94d0 0x55a56ddeb8c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddd9a80 0x55a56ddd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddd6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 --2- 172.21.15.67:0/1716388589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddcd1d0 0x55a56ddd6a30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 >> 172.21.15.67:0/1716388589 conn(0x55a56ddcb000 msgr2=0x55a56ddd0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.881+0000 7f37a4c00640 0 client.15735 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.882+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 shutdown_connections 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.882+0000 7f37c48bb740 1 -- 172.21.15.67:0/1716388589 wait complete. 2024-09-10T23:01:43.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.882+0000 7f37c48bb740 20 client.15735 trim_cache size 0 max 16384 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 Processor -- start 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 -- start start 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddc9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b4620 con 0x55a56ddd9a80 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b1ef0 con 0x55a56ddcd1d0 2024-09-10T23:01:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37c48bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1afd40 con 0x55a56ddcdaf0 2024-09-10T23:01:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.900+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49100/0 (socket says 172.21.15.67:49100) 2024-09-10T23:01:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 -- 172.21.15.67:0/920325199 learned_addr learned my addr 172.21.15.67:0/920325199 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37baa00640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37ba000640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddc9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 -- 172.21.15.67:0/920325199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 msgr2=0x55a56ddc9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddc9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 -- 172.21.15.67:0/920325199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 msgr2=0x55a56ddcc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 -- 172.21.15.67:0/920325199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a56ddd9210 con 0x55a56ddd9a80 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37ba000640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddc9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37baa00640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37bb400640 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37b40ca810 tx=0x7f37b4009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb3886e3eb65c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.901+0000 7f37a7400640 1 -- 172.21.15.67:0/920325199 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40ccef0 con 0x55a56ddd9a80 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37a7400640 1 -- 172.21.15.67:0/920325199 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b4036040 con 0x55a56ddd9a80 2024-09-10T23:01:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37a7400640 1 -- 172.21.15.67:0/920325199 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40ccd10 con 0x55a56ddd9a80 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 -- 172.21.15.67:0/920325199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 msgr2=0x55a56ddd3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37b40ca810 tx=0x7f37b4009980 comp rx=0 tx=0).stop 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 -- 172.21.15.67:0/920325199 shutdown_connections 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56ddd3c40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcdaf0 0x55a56ddc9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 --2- 172.21.15.67:0/920325199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcd1d0 0x55a56ddcc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 -- 172.21.15.67:0/920325199 >> 172.21.15.67:0/920325199 conn(0x55a56ddcb000 msgr2=0x55a56ddc98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 -- 172.21.15.67:0/920325199 shutdown_connections 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.902+0000 7f37c48bb740 1 -- 172.21.15.67:0/920325199 wait complete. 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 -- start start 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 Processor -- start 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcd1d0 0x55a56ddddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcdaf0 0x55a56ddde2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b03b0 con 0x55a56ddd9a80 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b1820 con 0x55a56ddcdaf0 2024-09-10T23:01:43.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.903+0000 7f37c48bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1b0a80 con 0x55a56ddcd1d0 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37c48bb740 10 client.0 _ll_register_callbacks cb 0x7f37a800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49114/0 (socket says 172.21.15.67:49114) 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 -- 172.21.15.67:0/7285192 learned_addr learned my addr 172.21.15.67:0/7285192 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37ba000640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcdaf0 0x55a56ddde2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37baa00640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcd1d0 0x55a56ddddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 -- 172.21.15.67:0/7285192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcd1d0 msgr2=0x55a56ddddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcd1d0 0x55a56ddddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 -- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcdaf0 msgr2=0x55a56ddde2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:43.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcdaf0 0x55a56ddde2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.904+0000 7f37bb400640 1 -- 172.21.15.67:0/7285192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a56de558c0 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37bb400640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37b40cd530 tx=0x7f37b4104da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc40770967ac0dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4004820 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f37b40cd800 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b4036040 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37b40cd0b0 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37c48bb740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a56de56b80 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.905+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37b4036040 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37b40d66a0 con 0x55a56ddd9a80 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:43.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 10 client.15741 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 20 client.15741 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56de57220 0x55a56de59640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 4 client.15741 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2ec564e5", "release": true, "await": true}] 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2ec564e5", "release": true, "await": true}) -- 0x55a56dd41c90 con 0x55a56de57220 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.906+0000 7f37baa00640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56de57220 0x55a56de59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.907+0000 7f37baa00640 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56de57220 0x55a56de59640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c17bc61447db25 server_cookie=6dc94d88bedefbd0 in_seq=0 out_seq=0 2024-09-10T23:01:43.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.907+0000 7f37a4c00640 10 client.15741 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:43.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37a4c00640 1 -- 172.21.15.67:0/7285192 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a56dd41c90 con 0x55a56de57220 2024-09-10T23:01:43.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37a4c00640 10 client.15741 handle_command_reply: tid=0 2024-09-10T23:01:43.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 client.15741 shutdown 2024-09-10T23:01:43.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 msgr2=0x55a56de55380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37b40cd530 tx=0x7f37b4104da0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 shutdown_connections 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55a56de57220 0x55a56de59640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a56ddd9a80 0x55a56de55380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a56ddcdaf0 0x55a56ddde2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 --2- 172.21.15.67:0/7285192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a56ddcd1d0 0x55a56ddddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 >> 172.21.15.67:0/7285192 conn(0x55a56ddcb000 msgr2=0x55a56ddca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.932+0000 7f37a4c00640 0 client.15741 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.933+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 shutdown_connections 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.933+0000 7f37c48bb740 1 -- 172.21.15.67:0/7285192 wait complete. 2024-09-10T23:01:43.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.933+0000 7f37c48bb740 20 client.15741 trim_cache size 0 max 16384 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 29, 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 8, 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "2ec564e5": { 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 8, 2024-09-10T23:01:43.935 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 140.5, 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 msgr2=0x7f37a00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 0x7f37a00b7a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a56ddcd180 tx=0x7f37a800b650 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 msgr2=0x7f37bc1a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37b4031040 tx=0x7f37b4032bc0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 shutdown_connections 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f37a00b5580 0x7f37a00b7a40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37bc10b180 0x7f37bc1a0a30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37bc10a800 0x7f37bc19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 --2- 172.21.15.67:0/554271580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37bc104df0 0x7f37bc19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 >> 172.21.15.67:0/554271580 conn(0x7f37bc100840 msgr2=0x7f37bc101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 shutdown_connections 2024-09-10T23:01:43.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:43.934+0000 7f37bbe00640 1 -- 172.21.15.67:0/554271580 wait complete. 2024-09-10T23:01:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:43 smithi067 ceph-mon[30939]: pgmap v472: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 130 KiB/s wr, 6 op/s 2024-09-10T23:01:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:43 smithi145 ceph-mon[34397]: pgmap v472: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 130 KiB/s wr, 6 op/s 2024-09-10T23:01:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:43 smithi092 ceph-mon[34223]: pgmap v472: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 130 KiB/s wr, 6 op/s 2024-09-10T23:01:44.885 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2ec564e5', total seconds elapsed: 24.7 2024-09-10T23:01:44.886 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-10T23:01:44.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:45.295 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 -- 172.21.15.67:0/1957134326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 msgr2=0x7fe36810f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 --2- 172.21.15.67:0/1957134326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36810f3d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ca0 tx=0x7fe36002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 -- 172.21.15.67:0/1957134326 shutdown_connections 2024-09-10T23:01:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 --2- 172.21.15.67:0/1957134326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36810f3d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 --2- 172.21.15.67:0/1957134326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 0x7fe36810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 --2- 172.21.15.67:0/1957134326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 0x7fe368103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 -- 172.21.15.67:0/1957134326 >> 172.21.15.67:0/1957134326 conn(0x7fe3680fd5b0 msgr2=0x7fe3680ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:45.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 -- 172.21.15.67:0/1957134326 shutdown_connections 2024-09-10T23:01:45.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.977+0000 7fe36e600640 1 -- 172.21.15.67:0/1957134326 wait complete. 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.978+0000 7fe36e600640 1 Processor -- start 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.978+0000 7fe36e600640 1 -- start start 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 0x7fe368198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 0x7fe368199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe368112ae0 con 0x7fe36810cf70 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe368112960 con 0x7fe368103b10 2024-09-10T23:01:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36e600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe368112c60 con 0x7fe368104430 2024-09-10T23:01:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49128/0 (socket says 172.21.15.67:49128) 2024-09-10T23:01:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36dc00640 1 -- 172.21.15.67:0/950384494 learned_addr learned my addr 172.21.15.67:0/950384494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe35fe00640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 0x7fe368199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.979+0000 7fe36d200640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 0x7fe368198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 -- 172.21.15.67:0/950384494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 msgr2=0x7fe368199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 0x7fe368199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 -- 172.21.15.67:0/950384494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 msgr2=0x7fe368198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 0x7fe368198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 -- 172.21.15.67:0/950384494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36819efc0 con 0x7fe36810cf70 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.980+0000 7fe36dc00640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe360031040 tx=0x7fe360032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.981+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004040 con 0x7fe36810cf70 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.981+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36819f250 con 0x7fe36810cf70 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.981+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a5970 con 0x7fe36810cf70 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.981+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe360004670 con 0x7fe36810cf70 2024-09-10T23:01:45.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.981+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360030bb0 con 0x7fe36810cf70 2024-09-10T23:01:45.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.982+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3440052c0 con 0x7fe36810cf70 2024-09-10T23:01:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.985+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe3600041e0 con 0x7fe36810cf70 2024-09-10T23:01:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.986+0000 7fe366000640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 0x7fe34c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.987+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fe360107520 con 0x7fe36810cf70 2024-09-10T23:01:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.987+0000 7fe36d200640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 0x7fe34c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:45.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.988+0000 7fe36d200640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 0x7fe34c0b7a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe358005d00 tx=0x7fe35800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:45.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:45.991+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe3600cdd10 con 0x7fe36810cf70 2024-09-10T23:01:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:45 smithi145 ceph-mon[34397]: pgmap v473: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-09-10T23:01:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.261+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe344005880 con 0x7fe36810cf70 2024-09-10T23:01:46.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:45 smithi067 ceph-mon[30939]: pgmap v473: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-09-10T23:01:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.262+0000 7fe366000640 1 -- 172.21.15.67:0/950384494 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7fe360004820 con 0x7fe36810cf70 2024-09-10T23:01:46.263 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:01:46.263 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-09-10T23:01:03:188481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:01:02.557752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:01:46.263 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 msgr2=0x7fe34c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 0x7fe34c0b7a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe358005d00 tx=0x7fe35800b410 comp rx=0 tx=0).stop 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 msgr2=0x7fe36819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe360031040 tx=0x7fe360032bc0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 shutdown_connections 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe34c0b5580 0x7fe34c0b7a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36810cf70 0x7fe36819e7f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe368104430 0x7fe368199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 --2- 172.21.15.67:0/950384494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe368103b10 0x7fe368198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 >> 172.21.15.67:0/950384494 conn(0x7fe3680fd5b0 msgr2=0x7fe36810f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 shutdown_connections 2024-09-10T23:01:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:46.265+0000 7fe36e600640 1 -- 172.21.15.67:0/950384494 wait complete. 2024-09-10T23:01:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:45 smithi092 ceph-mon[34223]: pgmap v473: 129 pgs: 129 active+clean; 233 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-09-10T23:01:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:46 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/950384494' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:46 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/950384494' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:47.162 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-10T23:01:47.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-10T23:01:47.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:46 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/950384494' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:47.587 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:47 smithi067 ceph-mon[30939]: pgmap v474: 129 pgs: 129 active+clean; 237 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 343 KiB/s wr, 7 op/s 2024-09-10T23:01:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:47 smithi145 ceph-mon[34397]: pgmap v474: 129 pgs: 129 active+clean; 237 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 343 KiB/s wr, 7 op/s 2024-09-10T23:01:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 -- 172.21.15.67:0/246906433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 msgr2=0x7facc4115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 --2- 172.21.15.67:0/246906433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc4115d50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7facbc009ca0 tx=0x7facbc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 -- 172.21.15.67:0/246906433 shutdown_connections 2024-09-10T23:01:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 --2- 172.21.15.67:0/246906433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc4115d50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 --2- 172.21.15.67:0/246906433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc4113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 --2- 172.21.15.67:0/246906433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 -- 172.21.15.67:0/246906433 >> 172.21.15.67:0/246906433 conn(0x7facc40fe6d0 msgr2=0x7facc4100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 -- 172.21.15.67:0/246906433 shutdown_connections 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.256+0000 7facc3200640 1 -- 172.21.15.67:0/246906433 wait complete. 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.257+0000 7facc3200640 1 Processor -- start 2024-09-10T23:01:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 -- start start 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc41a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7facc41190c0 con 0x7facc4113960 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc4118f40 con 0x7facc4075e60 2024-09-10T23:01:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facc4119240 con 0x7facc4075540 2024-09-10T23:01:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc1400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.258+0000 7facc1400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:34368/0 (socket says 172.21.15.67:34368) 2024-09-10T23:01:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1400640 1 -- 172.21.15.67:0/698704103 learned_addr learned my addr 172.21.15.67:0/698704103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc2800640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc41a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1e00640 1 -- 172.21.15.67:0/698704103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 msgr2=0x7facc419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1e00640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1e00640 1 -- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 msgr2=0x7facc41a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1e00640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc41a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1e00640 1 -- 172.21.15.67:0/698704103 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41a5600 con 0x7facc4075540 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.259+0000 7facc1400640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facc1e00640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc419f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facb000e9f0 tx=0x7facb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facbaa00640 1 -- 172.21.15.67:0/698704103 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000ccc0 con 0x7facc4075540 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc41a58f0 con 0x7facc4075540 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facbaa00640 1 -- 172.21.15.67:0/698704103 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb00108c0 con 0x7facc4075540 2024-09-10T23:01:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facbaa00640 1 -- 172.21.15.67:0/698704103 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00187e0 con 0x7facc4075540 2024-09-10T23:01:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.260+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41abfb0 con 0x7facc4075540 2024-09-10T23:01:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.263+0000 7faccb5c0740 1 Processor -- start 2024-09-10T23:01:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.263+0000 7faccb5c0740 1 -- start start 2024-09-10T23:01:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e52ba0 0x55ace1e54f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ace1dbbca0 con 0x55ace1e462f0 2024-09-10T23:01:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ace1dab0d0 con 0x55ace1e52ba0 2024-09-10T23:01:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7faccb5c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ace1daeee0 con 0x55ace1e46c10 2024-09-10T23:01:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49148/0 (socket says 172.21.15.67:49148) 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 -- 172.21.15.67:0/603842833 learned_addr learned my addr 172.21.15.67:0/603842833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 -- 172.21.15.67:0/603842833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 msgr2=0x55ace1e52660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e52660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 -- 172.21.15.67:0/603842833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e52ba0 msgr2=0x55ace1e54f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e52ba0 0x55ace1e54f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.264+0000 7facc2800640 1 -- 172.21.15.67:0/603842833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace1d5d400 con 0x55ace1e462f0 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.265+0000 7facbaa00640 1 -- 172.21.15.67:0/698704103 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7facb0010430 con 0x7facc4075540 2024-09-10T23:01:48.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.265+0000 7facc2800640 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7facc406c220 tx=0x7facbc005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8022488a950a2683 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.265+0000 7facaf000640 1 -- 172.21.15.67:0/603842833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0316d0 con 0x55ace1e462f0 2024-09-10T23:01:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.265+0000 7facaf000640 1 -- 172.21.15.67:0/603842833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facbc036030 con 0x55ace1e462f0 2024-09-10T23:01:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.265+0000 7facaf000640 1 -- 172.21.15.67:0/603842833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0042a0 con 0x55ace1e462f0 2024-09-10T23:01:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 -- 172.21.15.67:0/603842833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 msgr2=0x55ace1e466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7facc406c220 tx=0x7facbc005f50 comp rx=0 tx=0).stop 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 -- 172.21.15.67:0/603842833 shutdown_connections 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e52ba0 0x55ace1e54f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e52660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 --2- 172.21.15.67:0/603842833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e466d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.266+0000 7faccb5c0740 1 -- 172.21.15.67:0/603842833 >> 172.21.15.67:0/603842833 conn(0x55ace1e44120 msgr2=0x55ace1e452c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7faccb5c0740 1 -- 172.21.15.67:0/603842833 shutdown_connections 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7facbaa00640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 0x7faca80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7faccb5c0740 1 -- 172.21.15.67:0/603842833 wait complete. 2024-09-10T23:01:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7facbaa00640 1 -- 172.21.15.67:0/698704103 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7facb00e66d0 con 0x7facc4075540 2024-09-10T23:01:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7facc1400640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 0x7faca80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7faccb5c0740 1 -- start start 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.267+0000 7faccb5c0740 1 Processor -- start 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7facc1400640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 0x7faca80b78d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7facc4116030 tx=0x7facb40046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e462f0 0x55ace1e52550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e4f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ace1c0c7a0 con 0x55ace1e52ba0 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ace1d50a10 con 0x55ace1e462f0 2024-09-10T23:01:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ace1e55640 con 0x55ace1e46c10 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7faccb5c0740 10 client.0 _ll_register_callbacks cb 0x55ace1d60be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.268+0000 7facc2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc1e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e462f0 0x55ace1e52550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49156/0 (socket says 172.21.15.67:49156) 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 -- 172.21.15.67:0/1730124781 learned_addr learned my addr 172.21.15.67:0/1730124781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc1400640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e4f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:47 smithi092 ceph-mon[34223]: pgmap v474: 129 pgs: 129 active+clean; 237 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 343 KiB/s wr, 7 op/s 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 -- 172.21.15.67:0/1730124781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 msgr2=0x55ace1e4f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e4f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 -- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e462f0 msgr2=0x55ace1e52550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e462f0 0x55ace1e52550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.269+0000 7facc2800640 1 -- 172.21.15.67:0/1730124781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace1b18b90 con 0x55ace1e52ba0 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7facc2800640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7facbc004af0 tx=0x7facbc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88934af659f1e4be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faca3400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc04a040 con 0x55ace1e52ba0 2024-09-10T23:01:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facbc036030 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7facbc031c40 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7facbc05e050 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faccb5c0740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.270+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ace1e61050 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facbc036030 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7facbc04d030 con 0x55ace1e52ba0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 10 client.15759 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 20 client.15759 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1e623a0 0x55ace1e64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 4 client.15759 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.271+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ace1dbdbf0 con 0x55ace1e623a0 2024-09-10T23:01:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.272+0000 7facc1e00640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1e623a0 0x55ace1e64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.272+0000 7facc1e00640 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1e623a0 0x55ace1e64790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5259f06bd43a617 server_cookie=310da308285eeded in_seq=0 out_seq=0 2024-09-10T23:01:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.272+0000 7faca3400640 10 client.15759 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.273+0000 7faca3400640 1 -- 172.21.15.67:0/1730124781 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ace1dbdbf0 con 0x55ace1e623a0 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.273+0000 7faca3400640 10 client.15759 handle_command_reply: tid=0 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.273+0000 7faccb5c0740 1 client.15759 shutdown 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 msgr2=0x55ace1e50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7facbc004af0 tx=0x7facbc003c20 comp rx=0 tx=0).stop 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 shutdown_connections 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1e623a0 0x55ace1e64790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e50090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e46c10 0x55ace1e4f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1730124781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e462f0 0x55ace1e52550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 >> 172.21.15.67:0/1730124781 conn(0x55ace1e44120 msgr2=0x55ace1e49c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faca3400640 0 client.15759 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 shutdown_connections 2024-09-10T23:01:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 1 -- 172.21.15.67:0/1730124781 wait complete. 2024-09-10T23:01:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.274+0000 7faccb5c0740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T23:01:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 Processor -- start 2024-09-10T23:01:48.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 -- start start 2024-09-10T23:01:48.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e53f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 0x55ace1e544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b8940 con 0x55ace1e462f0 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b6210 con 0x55ace1e46c10 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7faccb5c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b4060 con 0x55ace1e52ba0 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7facc1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.295+0000 7facc1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49172/0 (socket says 172.21.15.67:49172) 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 -- 172.21.15.67:0/758926247 learned_addr learned my addr 172.21.15.67:0/758926247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:48.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1400640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e53f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc2800640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 0x55ace1e544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 -- 172.21.15.67:0/758926247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 msgr2=0x55ace1e544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 0x55ace1e544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 -- 172.21.15.67:0/758926247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 msgr2=0x55ace1e53f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e53f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 -- 172.21.15.67:0/758926247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace1e3f960 con 0x55ace1e462f0 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc2800640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 0x55ace1e544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.296+0000 7facc1e00640 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7facb00a8de0 tx=0x7facb00a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f643d57ae5dd754c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.297+0000 7facae600640 1 -- 172.21.15.67:0/758926247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00b5cf0 con 0x55ace1e462f0 2024-09-10T23:01:48.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.297+0000 7facae600640 1 -- 172.21.15.67:0/758926247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb00e5040 con 0x55ace1e462f0 2024-09-10T23:01:48.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.297+0000 7facae600640 1 -- 172.21.15.67:0/758926247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00e6460 con 0x55ace1e462f0 2024-09-10T23:01:48.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.297+0000 7faccb5c0740 1 -- 172.21.15.67:0/758926247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 msgr2=0x55ace1e4eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.297+0000 7faccb5c0740 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7facb00a8de0 tx=0x7facb00a8040 comp rx=0 tx=0).stop 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 -- 172.21.15.67:0/758926247 shutdown_connections 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e52ba0 0x55ace1e544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e53f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 --2- 172.21.15.67:0/758926247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e462f0 0x55ace1e4eed0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 -- 172.21.15.67:0/758926247 >> 172.21.15.67:0/758926247 conn(0x55ace1e44120 msgr2=0x55ace1e51810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 -- 172.21.15.67:0/758926247 shutdown_connections 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 -- 172.21.15.67:0/758926247 wait complete. 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.298+0000 7faccb5c0740 1 -- start start 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 Processor -- start 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 0x55ace1e4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b46d0 con 0x55ace1e52ba0 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b5b40 con 0x55ace1e46c10 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b4da0 con 0x55ace1e462f0 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7faccb5c0740 10 client.0 _ll_register_callbacks cb 0x7facbc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.299+0000 7facc1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 0x55ace1e4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 0x55ace1e4d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:48616/0 (socket says 172.21.15.67:48616) 2024-09-10T23:01:48.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc1e00640 1 -- 172.21.15.67:0/1210822544 learned_addr learned my addr 172.21.15.67:0/1210822544 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc1400640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 -- 172.21.15.67:0/1210822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 msgr2=0x55ace1e4d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 0x55ace1e4d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 -- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 msgr2=0x55ace1e5a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e5a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.300+0000 7facc2800640 1 -- 172.21.15.67:0/1210822544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace1e5b400 con 0x55ace1e52ba0 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.301+0000 7facc2800640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e5ae10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7facbc02ade0 tx=0x7facbc005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aab92a7e1a5afc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.301+0000 7faca3400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.302+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc031c40 con 0x55ace1e52ba0 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.302+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb00e5040 con 0x55ace1e52ba0 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.302+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7facbc031de0 con 0x55ace1e52ba0 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.302+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7facbc004170 con 0x55ace1e52ba0 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.302+0000 7faccb5c0740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-09-10T23:01:48.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.303+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ace1e4c9f0 con 0x55ace1e52ba0 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.303+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb00e5040 con 0x55ace1e52ba0 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7facbc04d040 con 0x55ace1e52ba0 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 10 client.15765 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 20 client.15765 populate_metadata read hostname 'smithi067' 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1ed0780 0x55ace1ed2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 4 client.15765 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.304+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x55ace1db96a0 con 0x55ace1ed0780 2024-09-10T23:01:48.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.305+0000 7facc1e00640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1ed0780 0x55ace1ed2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.306+0000 7facc1e00640 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1ed0780 0x55ace1ed2ba0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dbd402f908e950a server_cookie=8e98dc9ff1180410 in_seq=0 out_seq=0 2024-09-10T23:01:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.306+0000 7faca3400640 10 client.15765 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:48.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faca3400640 1 -- 172.21.15.67:0/1210822544 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ace1db96a0 con 0x55ace1ed0780 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faca3400640 10 client.15765 handle_command_reply: tid=0 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 client.15765 shutdown 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 msgr2=0x55ace1e5ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e5ae10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7facbc02ade0 tx=0x7facbc005df0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 shutdown_connections 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55ace1ed0780 0x55ace1ed2ba0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ace1e52ba0 0x55ace1e5ae10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ace1e46c10 0x55ace1e5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 --2- 172.21.15.67:0/1210822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ace1e462f0 0x55ace1e4d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 >> 172.21.15.67:0/1210822544 conn(0x55ace1e44120 msgr2=0x55ace1e493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faca3400640 0 client.15765 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:01:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 shutdown_connections 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.417+0000 7faccb5c0740 1 -- 172.21.15.67:0/1210822544 wait complete. 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7faccb5c0740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 29, 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 10, 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "67bda137": { 2024-09-10T23:01:48.420 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 10, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 140.5, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:01:48.421 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 msgr2=0x7faca80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 0x7faca80b78d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7facc4116030 tx=0x7facb40046c0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 msgr2=0x7facc419f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc419f550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facb000e9f0 tx=0x7facb000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 shutdown_connections 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faca80b5410 0x7faca80b78d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facc4113960 0x7facc41a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc4075e60 0x7facc419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 --2- 172.21.15.67:0/698704103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4075540 0x7facc419f550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.418+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 >> 172.21.15.67:0/698704103 conn(0x7facc40fe6d0 msgr2=0x7facc40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.419+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 shutdown_connections 2024-09-10T23:01:48.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:01:48.419+0000 7facc3200640 1 -- 172.21.15.67:0/698704103 wait complete. 2024-09-10T23:01:49.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '67bda137', quiesce took 4.4 seconds. Will release after: 16.1 2024-09-10T23:01:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:49 smithi092 ceph-mon[34223]: pgmap v475: 129 pgs: 129 active+clean; 248 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:01:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:49 smithi067 ceph-mon[30939]: pgmap v475: 129 pgs: 129 active+clean; 248 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:01:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:49 smithi145 ceph-mon[34397]: pgmap v475: 129 pgs: 129 active+clean; 248 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:01:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:51 smithi092 ceph-mon[34223]: pgmap v476: 129 pgs: 129 active+clean; 292 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.9 MiB/s wr, 8 op/s 2024-09-10T23:01:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:51 smithi067 ceph-mon[30939]: pgmap v476: 129 pgs: 129 active+clean; 292 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.9 MiB/s wr, 8 op/s 2024-09-10T23:01:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:51 smithi145 ceph-mon[34397]: pgmap v476: 129 pgs: 129 active+clean; 292 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.9 MiB/s wr, 8 op/s 2024-09-10T23:01:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:53 smithi092 ceph-mon[34223]: pgmap v477: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:01:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:53 smithi067 ceph-mon[30939]: pgmap v477: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:01:54.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:53 smithi145 ceph-mon[34397]: pgmap v477: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:01:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:55 smithi092 ceph-mon[34223]: pgmap v478: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-09-10T23:01:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:55 smithi067 ceph-mon[30939]: pgmap v478: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-09-10T23:01:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:55 smithi145 ceph-mon[34397]: pgmap v478: 129 pgs: 129 active+clean; 296 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-09-10T23:01:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:57 smithi092 ceph-mon[34223]: pgmap v479: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:01:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:57 smithi067 ceph-mon[30939]: pgmap v479: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:01:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:57 smithi145 ceph-mon[34397]: pgmap v479: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:02:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:01:59 smithi092 ceph-mon[34223]: pgmap v480: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:02:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:01:59 smithi067 ceph-mon[30939]: pgmap v480: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:02:00.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:01:59 smithi145 ceph-mon[34397]: pgmap v480: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-09-10T23:02:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:01 smithi092 ceph-mon[34223]: pgmap v481: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:02:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:01 smithi067 ceph-mon[30939]: pgmap v481: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:02:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:01 smithi145 ceph-mon[34397]: pgmap v481: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:02:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:03 smithi092 ceph-mon[34223]: pgmap v482: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 714 KiB/s wr, 19 op/s 2024-09-10T23:02:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:03 smithi067 ceph-mon[30939]: pgmap v482: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 714 KiB/s wr, 19 op/s 2024-09-10T23:02:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:03 smithi145 ceph-mon[34397]: pgmap v482: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 714 KiB/s wr, 19 op/s 2024-09-10T23:02:05.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:02:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:06.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:05 smithi067 ceph-mon[30939]: pgmap v483: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 384 KiB/s wr, 21 op/s 2024-09-10T23:02:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:05 smithi092 ceph-mon[34223]: pgmap v483: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 384 KiB/s wr, 21 op/s 2024-09-10T23:02:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:05 smithi145 ceph-mon[34397]: pgmap v483: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 384 KiB/s wr, 21 op/s 2024-09-10T23:02:06.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 -- 172.21.15.67:0/2702853596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 msgr2=0x7fcb0c1135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:06.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 --2- 172.21.15.67:0/2702853596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c1135e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009900 tx=0x7fcaf802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 -- 172.21.15.67:0/2702853596 shutdown_connections 2024-09-10T23:02:06.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 --2- 172.21.15.67:0/2702853596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c113b20 0x7fcb0c115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 --2- 172.21.15.67:0/2702853596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c1135e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 --2- 172.21.15.67:0/2702853596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c075540 0x7fcb0c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 -- 172.21.15.67:0/2702853596 >> 172.21.15.67:0/2702853596 conn(0x7fcb0c0fe830 msgr2=0x7fcb0c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.434+0000 7fcb14000640 1 -- 172.21.15.67:0/2702853596 shutdown_connections 2024-09-10T23:02:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.435+0000 7fcb14000640 1 -- 172.21.15.67:0/2702853596 wait complete. 2024-09-10T23:02:06.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.435+0000 7fcb14000640 1 Processor -- start 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 -- start start 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c113b20 0x7fcb0c1a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c1192b0 con 0x7fcb0c075e60 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c119130 con 0x7fcb0c113b20 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb14000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c119430 con 0x7fcb0c075540 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb10c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb11600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb10c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52632/0 (socket says 172.21.15.67:52632) 2024-09-10T23:02:06.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb11600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:60298/0 (socket says 172.21.15.67:60298) 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.436+0000 7fcb10c00640 1 -- 172.21.15.67:0/1872961374 learned_addr learned my addr 172.21.15.67:0/1872961374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb12000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c113b20 0x7fcb0c1a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 -- 172.21.15.67:0/1872961374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c113b20 msgr2=0x7fcb0c1a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c113b20 0x7fcb0c1a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 -- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 msgr2=0x7fcb0c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 -- 172.21.15.67:0/1872961374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a58f0 con 0x7fcb0c075540 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb10c00640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.437+0000 7fcb11600640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d6d0 tx=0x7fcb0000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.438+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00017070 con 0x7fcb0c075540 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.438+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb00009820 con 0x7fcb0c075540 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.438+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a5be0 con 0x7fcb0c075540 2024-09-10T23:02:06.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.438+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00004bc0 con 0x7fcb0c075540 2024-09-10T23:02:06.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.438+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1ac2a0 con 0x7fcb0c075540 2024-09-10T23:02:06.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.439+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c075920 con 0x7fcb0c075540 2024-09-10T23:02:06.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.441+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcb00005220 con 0x7fcb0c075540 2024-09-10T23:02:06.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.443+0000 7fcb0a000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 0x7fcaf40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:06.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.443+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fcb000e6220 con 0x7fcb0c075540 2024-09-10T23:02:06.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.443+0000 7fcb10c00640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 0x7fcaf40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:06.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.444+0000 7fcb10c00640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 0x7fcaf40b7910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80096c0 tx=0x7fcaf8005e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:06.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.444+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb00005540 con 0x7fcb0c075540 2024-09-10T23:02:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.678+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb0c1ac660 con 0x7fcb0c075540 2024-09-10T23:02:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.679+0000 7fcb0a000640 1 -- 172.21.15.67:0/1872961374 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7fcb000ac360 con 0x7fcb0c075540 2024-09-10T23:02:06.681 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:02:06.681 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-09-10T23:01:03:188481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:01:02.557752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:02:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-09-10T23:02:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 msgr2=0x7fcaf40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 0x7fcaf40b7910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80096c0 tx=0x7fcaf8005e60 comp rx=0 tx=0).stop 2024-09-10T23:02:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 msgr2=0x7fcb0c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d6d0 tx=0x7fcb0000db90 comp rx=0 tx=0).stop 2024-09-10T23:02:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 shutdown_connections 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fcaf40b5450 0x7fcaf40b7910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb0c113b20 0x7fcb0c1a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb0c075e60 0x7fcb0c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 --2- 172.21.15.67:0/1872961374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb0c075540 0x7fcb0c19f840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.681+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 >> 172.21.15.67:0/1872961374 conn(0x7fcb0c0fe830 msgr2=0x7fcb0c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.682+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 shutdown_connections 2024-09-10T23:02:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:06.682+0000 7fcb14000640 1 -- 172.21.15.67:0/1872961374 wait complete. 2024-09-10T23:02:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:06 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1872961374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:07.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:06 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1872961374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:07.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:06 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1872961374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:07.576 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 67bda137 --release --await' 2024-09-10T23:02:07.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 67bda137 --release --await 2024-09-10T23:02:07.957 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:07 smithi092 ceph-mon[34223]: pgmap v484: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 383 KiB/s wr, 22 op/s 2024-09-10T23:02:08.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:07 smithi067 ceph-mon[30939]: pgmap v484: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 383 KiB/s wr, 22 op/s 2024-09-10T23:02:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:07 smithi145 ceph-mon[34397]: pgmap v484: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 383 KiB/s wr, 22 op/s 2024-09-10T23:02:08.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 -- 172.21.15.67:0/3848592580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08109150 msgr2=0x7f6d0810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 --2- 172.21.15.67:0/3848592580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08109150 0x7f6d0810f950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009ca0 tx=0x7f6d0002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 -- 172.21.15.67:0/3848592580 shutdown_connections 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 --2- 172.21.15.67:0/3848592580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08109150 0x7f6d0810f950 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 --2- 172.21.15.67:0/3848592580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d081087d0 0x7f6d08108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 --2- 172.21.15.67:0/3848592580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08102dc0 0x7f6d081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.556+0000 7f6d10600640 1 -- 172.21.15.67:0/3848592580 >> 172.21.15.67:0/3848592580 conn(0x7f6d080fe830 msgr2=0x7f6d08100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.557+0000 7f6d10600640 1 -- 172.21.15.67:0/3848592580 shutdown_connections 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.557+0000 7f6d10600640 1 -- 172.21.15.67:0/3848592580 wait complete. 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 Processor -- start 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 -- start start 2024-09-10T23:02:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d081087d0 0x7f6d081996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d08109150 0x7f6d0819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112cd0 con 0x7f6d08102dc0 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112b50 con 0x7f6d081087d0 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d10600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112e50 con 0x7f6d08109150 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52652/0 (socket says 172.21.15.67:52652) 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d0dc00640 1 -- 172.21.15.67:0/2355050075 learned_addr learned my addr 172.21.15.67:0/2355050075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.558+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d081087d0 0x7f6d081996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d08109150 0x7f6d0819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 -- 172.21.15.67:0/2355050075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d08109150 msgr2=0x7f6d0819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d08109150 0x7f6d0819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 -- 172.21.15.67:0/2355050075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d081087d0 msgr2=0x7f6d081996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d081087d0 0x7f6d081996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 -- 172.21.15.67:0/2355050075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0819f250 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00d950 tx=0x7f6cfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.559+0000 7f6d06000640 1 -- 172.21.15.67:0/2355050075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc017070 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.560+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0819f540 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.560+0000 7f6d06000640 1 -- 172.21.15.67:0/2355050075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cfc009c30 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.560+0000 7f6d06000640 1 -- 172.21.15.67:0/2355050075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00f620 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.560+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d081a5dd0 con 0x7f6d08102dc0 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 Processor -- start 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 -- start start 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x561131314770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x5611313170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.563+0000 7f6d12fea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56113127b290 con 0x561131303ff0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d12fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56113126a6c0 con 0x561131314cb0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d12fea740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56113126e4d0 con 0x561131304910 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d06000640 1 -- 172.21.15.67:0/2355050075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6cfc010460 con 0x7f6d08102dc0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52654/0 (socket says 172.21.15.67:52654) 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 -- 172.21.15.67:0/5475078 learned_addr learned my addr 172.21.15.67:0/5475078 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x561131314770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0d200640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x5611313170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 -- 172.21.15.67:0/5475078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 msgr2=0x561131314770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x561131314770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 -- 172.21.15.67:0/5475078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 msgr2=0x5611313170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x5611313170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0e600640 1 -- 172.21.15.67:0/5475078 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611311f7460 con 0x561131303ff0 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.564+0000 7f6d0d200640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x5611313170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.565+0000 7f6d0e600640 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009ca0 tx=0x7f6d00031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b426e6d4fb607e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.565+0000 7f6cfaa00640 1 -- 172.21.15.67:0/5475078 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00031980 con 0x561131303ff0 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.565+0000 7f6cfaa00640 1 -- 172.21.15.67:0/5475078 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d00036070 con 0x561131303ff0 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.565+0000 7f6cfaa00640 1 -- 172.21.15.67:0/5475078 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004520 con 0x561131303ff0 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d06000640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 0x7f6cf00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 -- 172.21.15.67:0/5475078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 msgr2=0x5611313043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009ca0 tx=0x7f6d00031450 comp rx=0 tx=0).stop 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 -- 172.21.15.67:0/5475078 shutdown_connections 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x5611313170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x561131314770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 --2- 172.21.15.67:0/5475078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x5611313043d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d06000640 1 -- 172.21.15.67:0/2355050075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f6cfc0e5ee0 con 0x7f6d08102dc0 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d12fea740 1 -- 172.21.15.67:0/5475078 >> 172.21.15.67:0/5475078 conn(0x561131301e20 msgr2=0x561131302fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.566+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 0x7f6cf00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.567+0000 7f6d12fea740 1 -- 172.21.15.67:0/5475078 shutdown_connections 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.567+0000 7f6d12fea740 1 -- 172.21.15.67:0/5475078 wait complete. 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.567+0000 7f6d0dc00640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 0x7f6cf00b7990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561131303ad0 tx=0x7f6cfc000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.567+0000 7f6d12fea740 1 -- start start 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.567+0000 7f6d12fea740 1 Processor -- start 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131311d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x5611313127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5611310cbbb0 con 0x561131304910 2024-09-10T23:02:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5611312056c0 con 0x561131303ff0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561131317780 con 0x561131314cb0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d12fea740 10 client.0 _ll_register_callbacks cb 0x561131220490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d0d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d0d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52664/0 (socket says 172.21.15.67:52664) 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d0d200640 1 -- 172.21.15.67:0/2836026217 learned_addr learned my addr 172.21.15.67:0/2836026217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.568+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x5611313127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131311d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 -- 172.21.15.67:0/2836026217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 msgr2=0x5611313127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x5611313127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 -- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 msgr2=0x561131311d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131311d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 -- 172.21.15.67:0/2836026217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561130fc8b00 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400ba70 tx=0x7f6cf400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842f4b9e699ac25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6cef400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf401b640 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f6cf401b7e0 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.569+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00036070 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cf4024050 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56113103f960 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d00036070 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cf4019020 con 0x561131304910 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 10 client.15783 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 20 client.15783 populate_metadata read hostname 'smithi067' 2024-09-10T23:02:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5611313247a0 0x561131326b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 4 client.15783 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.570+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113127d1e0 con 0x5611313247a0 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.571+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5611313247a0 0x561131326b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.571+0000 7f6cef400640 10 client.15783 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.571+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5611313247a0 0x561131326b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55229f8153a50cc server_cookie=8870a52915161059 in_seq=0 out_seq=0 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6cef400640 1 -- 172.21.15.67:0/2836026217 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56113127d1e0 con 0x5611313247a0 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6cef400640 10 client.15783 handle_command_reply: tid=0 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 client.15783 shutdown 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 msgr2=0x5611313122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400ba70 tx=0x7f6cf400bf30 comp rx=0 tx=0).stop 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 shutdown_connections 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5611313247a0 0x561131326b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x5611313127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6cef400640 0 client.15783 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313122a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2836026217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131311d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.572+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 >> 172.21.15.67:0/2836026217 conn(0x561131301e20 msgr2=0x561131307950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.573+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 shutdown_connections 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.573+0000 7f6d12fea740 1 -- 172.21.15.67:0/2836026217 wait complete. 2024-09-10T23:02:08.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.573+0000 7f6d12fea740 20 client.15783 trim_cache size 0 max 16384 2024-09-10T23:02:08.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.591+0000 7f6d12fea740 1 Processor -- start 2024-09-10T23:02:08.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.591+0000 7f6d12fea740 1 -- start start 2024-09-10T23:02:08.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131310c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x561131313a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081b0430 con 0x561131304910 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081add00 con 0x561131303ff0 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d12fea740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081abb50 con 0x561131314cb0 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d0d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d0d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52666/0 (socket says 172.21.15.67:52666) 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d0d200640 1 -- 172.21.15.67:0/869738701 learned_addr learned my addr 172.21.15.67:0/869738701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d0e600640 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131310c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.592+0000 7f6d0e600640 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x561131313a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 -- 172.21.15.67:0/869738701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 msgr2=0x561131313a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x561131313a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 -- 172.21.15.67:0/869738701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 msgr2=0x561131310c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131310c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 -- 172.21.15.67:0/869738701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113130a150 con 0x561131304910 2024-09-10T23:02:08.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6d0d200640 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011a50 tx=0x7f6cf4005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a125e45ef8d6ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6cfa000640 1 -- 172.21.15.67:0/869738701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf402c020 con 0x561131304910 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6cfa000640 1 -- 172.21.15.67:0/869738701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf4005040 con 0x561131304910 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.593+0000 7f6cfa000640 1 -- 172.21.15.67:0/869738701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40048a0 con 0x561131304910 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 -- 172.21.15.67:0/869738701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 msgr2=0x5611313015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011a50 tx=0x7f6cf4005b90 comp rx=0 tx=0).stop 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 -- 172.21.15.67:0/869738701 shutdown_connections 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131314cb0 0x561131313a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131304910 0x5611313015a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 --2- 172.21.15.67:0/869738701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131303ff0 0x561131310c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 -- 172.21.15.67:0/869738701 >> 172.21.15.67:0/869738701 conn(0x561131301e20 msgr2=0x561131313340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 -- 172.21.15.67:0/869738701 shutdown_connections 2024-09-10T23:02:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.594+0000 7f6d12fea740 1 -- 172.21.15.67:0/869738701 wait complete. 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.595+0000 7f6d12fea740 1 -- start start 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.595+0000 7f6d12fea740 1 Processor -- start 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x56113132cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x56113131ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x56113131cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ac1c0 con 0x561131303ff0 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ad630 con 0x561131314cb0 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081ac890 con 0x561131304910 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d12fea740 10 client.0 _ll_register_callbacks cb 0x7f6cf4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d0e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x56113132cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d0e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x56113132cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52668/0 (socket says 172.21.15.67:52668) 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d0e600640 1 -- 172.21.15.67:0/2421886329 learned_addr learned my addr 172.21.15.67:0/2421886329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:02:08.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.596+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x56113131ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x56113131cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 -- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 msgr2=0x56113131cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x56113131cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 -- 172.21.15.67:0/2421886329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 msgr2=0x56113132cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x56113132cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 -- 172.21.15.67:0/2421886329 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113131d6c0 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x56113131cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d0d200640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x56113131ca40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011b40 tx=0x7f6cf4011670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3fe1a188ed1e0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6cef400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4035070 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf4005040 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.597+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f6cf4004660 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cf403e050 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 10 client.25519 fetch_fsmap learned FSMap version 29 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611313926d0 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf4005040 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cf4038020 con 0x561131304910 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 10 client.25519 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 10 client.25519 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:02:08.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 20 client.25519 populate_metadata read hostname 'smithi067' 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561131392d80 0x5611313951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 4 client.25519 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "67bda137", "release": true, "await": true}] 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.598+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "67bda137", "release": true, "await": true}) -- 0x561131278c90 con 0x561131392d80 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.599+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561131392d80 0x5611313951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.600+0000 7f6d0e600640 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561131392d80 0x5611313951a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491e114485885ce6 server_cookie=805dade10b10b541 in_seq=0 out_seq=0 2024-09-10T23:02:08.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.600+0000 7f6cef400640 10 client.25519 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6cef400640 1 -- 172.21.15.67:0/2421886329 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561131278c90 con 0x561131392d80 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6cef400640 10 client.25519 handle_command_reply: tid=0 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 client.25519 shutdown 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 msgr2=0x56113131ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x56113131ca40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4011b40 tx=0x7f6cf4011670 comp rx=0 tx=0).stop 2024-09-10T23:02:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 shutdown_connections 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561131392d80 0x5611313951a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561131314cb0 0x56113131cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561131304910 0x56113131ca40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 --2- 172.21.15.67:0/2421886329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561131303ff0 0x56113132cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 >> 172.21.15.67:0/2421886329 conn(0x561131301e20 msgr2=0x561131309960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.623+0000 7f6cef400640 0 client.25519 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.624+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 shutdown_connections 2024-09-10T23:02:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.624+0000 7f6d12fea740 1 -- 172.21.15.67:0/2421886329 wait complete. 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.624+0000 7f6d12fea740 20 client.25519 trim_cache size 0 max 16384 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 29, 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 12, 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "67bda137": { 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 12, 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:02:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 140.5, 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:02:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 msgr2=0x7f6cf00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 0x7f6cf00b7990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561131303ad0 tx=0x7f6cfc000bc0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 msgr2=0x7f6d081991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00d950 tx=0x7f6cfc00de10 comp rx=0 tx=0).stop 2024-09-10T23:02:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 shutdown_connections 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f6cf00b54d0 0x7f6cf00b7990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d08109150 0x7f6d0819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d081087d0 0x7f6d081996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 --2- 172.21.15.67:0/2355050075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d08102dc0 0x7f6d081991a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 >> 172.21.15.67:0/2355050075 conn(0x7f6d080fe830 msgr2=0x7f6d080ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 shutdown_connections 2024-09-10T23:02:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:02:08.625+0000 7f6d10600640 1 -- 172.21.15.67:0/2355050075 wait complete. 2024-09-10T23:02:09.596 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '67bda137', total seconds elapsed: 24.7 2024-09-10T23:02:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:09 smithi092 ceph-mon[34223]: pgmap v485: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-09-10T23:02:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:09 smithi067 ceph-mon[30939]: pgmap v485: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-09-10T23:02:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:09 smithi145 ceph-mon[34397]: pgmap v485: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-09-10T23:02:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:11 smithi092 ceph-mon[34223]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-09-10T23:02:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:11 smithi067 ceph-mon[30939]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-09-10T23:02:12.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:11 smithi145 ceph-mon[34397]: pgmap v486: 129 pgs: 129 active+clean; 300 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-09-10T23:02:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:13 smithi092 ceph-mon[34223]: pgmap v487: 129 pgs: 129 active+clean; 308 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 705 KiB/s wr, 21 op/s 2024-09-10T23:02:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:13 smithi067 ceph-mon[30939]: pgmap v487: 129 pgs: 129 active+clean; 308 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 705 KiB/s wr, 21 op/s 2024-09-10T23:02:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:13 smithi145 ceph-mon[34397]: pgmap v487: 129 pgs: 129 active+clean; 308 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 705 KiB/s wr, 21 op/s 2024-09-10T23:02:16.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:15 smithi092 ceph-mon[34223]: pgmap v488: 129 pgs: 129 active+clean; 344 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-09-10T23:02:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:15 smithi067 ceph-mon[30939]: pgmap v488: 129 pgs: 129 active+clean; 344 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-09-10T23:02:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:15 smithi145 ceph-mon[34397]: pgmap v488: 129 pgs: 129 active+clean; 344 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-09-10T23:02:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:17 smithi092 ceph-mon[34223]: pgmap v489: 129 pgs: 129 active+clean; 376 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-10T23:02:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:17 smithi067 ceph-mon[30939]: pgmap v489: 129 pgs: 129 active+clean; 376 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-10T23:02:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:17 smithi145 ceph-mon[34397]: pgmap v489: 129 pgs: 129 active+clean; 376 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-09-10T23:02:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:19 smithi092 ceph-mon[34223]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.0 MiB/s wr, 8 op/s 2024-09-10T23:02:20.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:19 smithi067 ceph-mon[30939]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.0 MiB/s wr, 8 op/s 2024-09-10T23:02:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:19 smithi145 ceph-mon[34397]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7.0 MiB/s wr, 8 op/s 2024-09-10T23:02:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:21 smithi092 ceph-mon[34223]: pgmap v491: 129 pgs: 129 active+clean; 444 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:02:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:21 smithi067 ceph-mon[30939]: pgmap v491: 129 pgs: 129 active+clean; 444 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:02:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:21 smithi145 ceph-mon[34397]: pgmap v491: 129 pgs: 129 active+clean; 444 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:02:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:23 smithi092 ceph-mon[34223]: pgmap v492: 129 pgs: 129 active+clean; 460 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:23 smithi067 ceph-mon[30939]: pgmap v492: 129 pgs: 129 active+clean; 460 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:23 smithi145 ceph-mon[34397]: pgmap v492: 129 pgs: 129 active+clean; 460 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:25 smithi092 ceph-mon[34223]: pgmap v493: 129 pgs: 129 active+clean; 492 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:02:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:25 smithi067 ceph-mon[30939]: pgmap v493: 129 pgs: 129 active+clean; 492 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:02:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:25 smithi145 ceph-mon[34397]: pgmap v493: 129 pgs: 129 active+clean; 492 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:02:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:27 smithi092 ceph-mon[34223]: pgmap v494: 129 pgs: 129 active+clean; 532 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:28.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:27 smithi067 ceph-mon[30939]: pgmap v494: 129 pgs: 129 active+clean; 532 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:27 smithi145 ceph-mon[34397]: pgmap v494: 129 pgs: 129 active+clean; 532 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:29 smithi092 ceph-mon[34223]: pgmap v495: 129 pgs: 129 active+clean; 548 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:30.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:29 smithi067 ceph-mon[30939]: pgmap v495: 129 pgs: 129 active+clean; 548 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:29 smithi145 ceph-mon[34397]: pgmap v495: 129 pgs: 129 active+clean; 548 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:31 smithi092 ceph-mon[34223]: pgmap v496: 129 pgs: 129 active+clean; 604 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:02:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:31 smithi067 ceph-mon[30939]: pgmap v496: 129 pgs: 129 active+clean; 604 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:02:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:31 smithi145 ceph-mon[34397]: pgmap v496: 129 pgs: 129 active+clean; 604 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:02:34.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:33 smithi092 ceph-mon[34223]: pgmap v497: 129 pgs: 129 active+clean; 608 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:33 smithi067 ceph-mon[30939]: pgmap v497: 129 pgs: 129 active+clean; 608 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:33 smithi145 ceph-mon[34397]: pgmap v497: 129 pgs: 129 active+clean; 608 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:02:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:35 smithi092 ceph-mon[34223]: pgmap v498: 129 pgs: 129 active+clean; 648 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:35 smithi067 ceph-mon[30939]: pgmap v498: 129 pgs: 129 active+clean; 648 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:35 smithi145 ceph-mon[34397]: pgmap v498: 129 pgs: 129 active+clean; 648 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:37 smithi092 ceph-mon[34223]: pgmap v499: 129 pgs: 129 active+clean; 688 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:37 smithi067 ceph-mon[30939]: pgmap v499: 129 pgs: 129 active+clean; 688 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:38.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:37 smithi145 ceph-mon[34397]: pgmap v499: 129 pgs: 129 active+clean; 688 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:39.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:38 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:39.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:38 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:39.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:38 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:39 smithi092 ceph-mon[34223]: pgmap v500: 129 pgs: 129 active+clean; 704 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:02:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:02:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:39 smithi067 ceph-mon[30939]: pgmap v500: 129 pgs: 129 active+clean; 704 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:02:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:02:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:39 smithi145 ceph-mon[34397]: pgmap v500: 129 pgs: 129 active+clean; 704 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:02:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:02:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:41 smithi092 ceph-mon[34223]: pgmap v501: 129 pgs: 129 active+clean; 756 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:02:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:41 smithi067 ceph-mon[30939]: pgmap v501: 129 pgs: 129 active+clean; 756 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:02:42.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:41 smithi145 ceph-mon[34397]: pgmap v501: 129 pgs: 129 active+clean; 756 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:02:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:43 smithi092 ceph-mon[34223]: pgmap v502: 129 pgs: 129 active+clean; 772 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:02:44.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:43 smithi067 ceph-mon[30939]: pgmap v502: 129 pgs: 129 active+clean; 772 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:02:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:43 smithi145 ceph-mon[34397]: pgmap v502: 129 pgs: 129 active+clean; 772 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:02:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:46 smithi092 ceph-mon[34223]: pgmap v503: 129 pgs: 129 active+clean; 800 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:46 smithi067 ceph-mon[30939]: pgmap v503: 129 pgs: 129 active+clean; 800 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:46 smithi145 ceph-mon[34397]: pgmap v503: 129 pgs: 129 active+clean; 800 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:48 smithi092 ceph-mon[34223]: pgmap v504: 129 pgs: 129 active+clean; 836 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:48 smithi067 ceph-mon[30939]: pgmap v504: 129 pgs: 129 active+clean; 836 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:48 smithi145 ceph-mon[34397]: pgmap v504: 129 pgs: 129 active+clean; 836 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:02:49.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:49 smithi145 ceph-mon[34397]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:49.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:49 smithi092 ceph-mon[34223]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:49.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:49 smithi067 ceph-mon[30939]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:51.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:51 smithi092 ceph-mon[34223]: pgmap v506: 129 pgs: 129 active+clean; 900 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:02:51.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:51 smithi067 ceph-mon[30939]: pgmap v506: 129 pgs: 129 active+clean; 900 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:02:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:51 smithi145 ceph-mon[34397]: pgmap v506: 129 pgs: 129 active+clean; 900 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:02:53.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:53 smithi092 ceph-mon[34223]: pgmap v507: 129 pgs: 129 active+clean; 908 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:53.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:53 smithi067 ceph-mon[30939]: pgmap v507: 129 pgs: 129 active+clean; 908 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:53 smithi145 ceph-mon[34397]: pgmap v507: 129 pgs: 129 active+clean; 908 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:02:55.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:55 smithi092 ceph-mon[34223]: pgmap v508: 129 pgs: 129 active+clean; 956 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:02:55.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:55 smithi067 ceph-mon[30939]: pgmap v508: 129 pgs: 129 active+clean; 956 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:02:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:55 smithi145 ceph-mon[34397]: pgmap v508: 129 pgs: 129 active+clean; 956 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:02:57.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:57 smithi092 ceph-mon[34223]: pgmap v509: 129 pgs: 129 active+clean; 992 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:57.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:57 smithi067 ceph-mon[30939]: pgmap v509: 129 pgs: 129 active+clean; 992 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:57 smithi145 ceph-mon[34397]: pgmap v509: 129 pgs: 129 active+clean; 992 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:02:59.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:02:59 smithi092 ceph-mon[34223]: pgmap v510: 129 pgs: 129 active+clean; 1008 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:02:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:02:59 smithi067 ceph-mon[30939]: pgmap v510: 129 pgs: 129 active+clean; 1008 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:02:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:02:59 smithi145 ceph-mon[34397]: pgmap v510: 129 pgs: 129 active+clean; 1008 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:03:01.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:01 smithi092 ceph-mon[34223]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:03:01.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:01 smithi067 ceph-mon[30939]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:03:01.632 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:01 smithi145 ceph-mon[34397]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:03:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:03 smithi092 ceph-mon[34223]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-09-10T23:03:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:03 smithi067 ceph-mon[30939]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-09-10T23:03:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:03 smithi145 ceph-mon[34397]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-09-10T23:03:05.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:05 smithi092 ceph-mon[34223]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 28 op/s 2024-09-10T23:03:05.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:05 smithi067 ceph-mon[30939]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 28 op/s 2024-09-10T23:03:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:05 smithi145 ceph-mon[34397]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 28 op/s 2024-09-10T23:03:07.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:07 smithi092 ceph-mon[34223]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:03:07.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:07 smithi067 ceph-mon[30939]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:03:07.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:07 smithi145 ceph-mon[34397]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:03:09.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:09 smithi092 ceph-mon[34223]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:09.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:09 smithi067 ceph-mon[30939]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:09.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:09 smithi145 ceph-mon[34397]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:11.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:11 smithi092 ceph-mon[34223]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:03:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:11 smithi067 ceph-mon[30939]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:03:11.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:11 smithi145 ceph-mon[34397]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:03:13.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:13 smithi092 ceph-mon[34223]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:13.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:13 smithi067 ceph-mon[30939]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:13.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:13 smithi145 ceph-mon[34397]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:03:15.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:15 smithi092 ceph-mon[34223]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:03:15.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:15 smithi067 ceph-mon[30939]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:03:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:15 smithi145 ceph-mon[34397]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:03:17.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:17 smithi092 ceph-mon[34223]: pgmap v519: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:17.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:17 smithi067 ceph-mon[30939]: pgmap v519: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:17 smithi145 ceph-mon[34397]: pgmap v519: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:19.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:19 smithi067 ceph-mon[30939]: pgmap v520: 129 pgs: 129 active+clean; 1.3 GiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:03:19.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:19 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:19 smithi145 ceph-mon[34397]: pgmap v520: 129 pgs: 129 active+clean; 1.3 GiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:03:19.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:19 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:19 smithi092 ceph-mon[34223]: pgmap v520: 129 pgs: 129 active+clean; 1.3 GiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:03:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:19 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:21.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:21 smithi067 ceph-mon[30939]: pgmap v521: 129 pgs: 129 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:03:21.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:21 smithi145 ceph-mon[34397]: pgmap v521: 129 pgs: 129 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:03:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:21 smithi092 ceph-mon[34223]: pgmap v521: 129 pgs: 129 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:03:23.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:23 smithi067 ceph-mon[30939]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:03:23.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:23 smithi145 ceph-mon[34397]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:03:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:23 smithi092 ceph-mon[34223]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:03:25.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:25 smithi067 ceph-mon[30939]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:25.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:25 smithi145 ceph-mon[34397]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:25.739 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:03:25.740 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:03:25.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:25 smithi092 ceph-mon[34223]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:03:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 -- 172.21.15.67:0/1540742849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 msgr2=0x7fd2e4111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 --2- 172.21.15.67:0/1540742849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e4111b30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009ca0 tx=0x7fd2dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 -- 172.21.15.67:0/1540742849 shutdown_connections 2024-09-10T23:03:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 --2- 172.21.15.67:0/1540742849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e4111b30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 --2- 172.21.15.67:0/1540742849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 0x7fd2e410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 --2- 172.21.15.67:0/1540742849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 0x7fd2e4105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 -- 172.21.15.67:0/1540742849 >> 172.21.15.67:0/1540742849 conn(0x7fd2e4100a50 msgr2=0x7fd2e4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 -- 172.21.15.67:0/1540742849 shutdown_connections 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.798+0000 7fd2eb800640 1 -- 172.21.15.67:0/1540742849 wait complete. 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.799+0000 7fd2eb800640 1 Processor -- start 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.799+0000 7fd2eb800640 1 -- start start 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.799+0000 7fd2eb800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 0x7fd2e419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.799+0000 7fd2eb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 0x7fd2e419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2eb800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2eb800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4114e60 con 0x7fd2e410b330 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2eb800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4114ce0 con 0x7fd2e4104fa0 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2eb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4114fe0 con 0x7fd2e410a9b0 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51790/0 (socket says 172.21.15.67:51790) 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 -- 172.21.15.67:0/2849985220 learned_addr learned my addr 172.21.15.67:0/2849985220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e3e00640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 0x7fd2e419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:26.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e8e00640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 0x7fd2e419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 -- 172.21.15.67:0/2849985220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 msgr2=0x7fd2e419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 0x7fd2e419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 -- 172.21.15.67:0/2849985220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 msgr2=0x7fd2e419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.800+0000 7fd2e9800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 0x7fd2e419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2e9800640 1 -- 172.21.15.67:0/2849985220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41a1400 con 0x7fd2e410b330 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2e9800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005cc0 tx=0x7fd2dc032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc032e00 con 0x7fd2e410b330 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2dc0044d0 con 0x7fd2e410b330 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc030b70 con 0x7fd2e410b330 2024-09-10T23:03:26.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41a1690 con 0x7fd2e410b330 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.801+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a7db0 con 0x7fd2e410b330 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.803+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4105380 con 0x7fd2e410b330 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.805+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd2dc004680 con 0x7fd2e410b330 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.807+0000 7fd2e1600640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 0x7fd2cc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.807+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fd2dc107600 con 0x7fd2e410b330 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.808+0000 7fd2e8e00640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 0x7fd2cc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.808+0000 7fd2e8e00640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 0x7fd2cc0b7910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800abe0 tx=0x7fd2d800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:26.812+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd2dc0cdd90 con 0x7fd2e410b330 2024-09-10T23:03:27.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.063+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2e410cb80 con 0x7fd2e410b330 2024-09-10T23:03:27.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.064+0000 7fd2e1600640 1 -- 172.21.15.67:0/2849985220 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11904 (secure 0 0 0) 0x7fd2dc0cd690 con 0x7fd2e410b330 2024-09-10T23:03:27.070 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:03:27.070 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-09-10T23:03:18:265861+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:17.560086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T23:03:27.070 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 msgr2=0x7fd2cc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 0x7fd2cc0b7910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800abe0 tx=0x7fd2d800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 msgr2=0x7fd2e41a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005cc0 tx=0x7fd2dc032a10 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 shutdown_connections 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2cc0b5450 0x7fd2cc0b7910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2e410b330 0x7fd2e41a0c30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e410a9b0 0x7fd2e419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 --2- 172.21.15.67:0/2849985220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2e4104fa0 0x7fd2e419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 >> 172.21.15.67:0/2849985220 conn(0x7fd2e4100a50 msgr2=0x7fd2e4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 shutdown_connections 2024-09-10T23:03:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:27.067+0000 7fd2eb800640 1 -- 172.21.15.67:0/2849985220 wait complete. 2024-09-10T23:03:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:27 smithi067 ceph-mon[30939]: pgmap v524: 129 pgs: 129 active+clean; 1.4 GiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:03:27.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:27 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2849985220' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:27 smithi145 ceph-mon[34397]: pgmap v524: 129 pgs: 129 active+clean; 1.4 GiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:03:27.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:27 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2849985220' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:27 smithi092 ceph-mon[34223]: pgmap v524: 129 pgs: 129 active+clean; 1.4 GiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:03:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:27 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2849985220' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:28.200 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:03:28.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:03:28.618 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 -- 172.21.15.67:0/1505330965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab8101fc0 msgr2=0x7faab810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 --2- 172.21.15.67:0/1505330965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab8101fc0 0x7faab810ed10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009900 tx=0x7faaa402a480 comp rx=0 tx=0).stop 2024-09-10T23:03:29.346 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:29 smithi067 ceph-mon[30939]: pgmap v525: 129 pgs: 129 active+clean; 1.4 GiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 -- 172.21.15.67:0/1505330965 shutdown_connections 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 --2- 172.21.15.67:0/1505330965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab810f250 0x7faab8111640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 --2- 172.21.15.67:0/1505330965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab8101fc0 0x7faab810ed10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 --2- 172.21.15.67:0/1505330965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab81016a0 0x7faab8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 -- 172.21.15.67:0/1505330965 >> 172.21.15.67:0/1505330965 conn(0x7faab80fd5b0 msgr2=0x7faab80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 -- 172.21.15.67:0/1505330965 shutdown_connections 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.344+0000 7faab7200640 1 -- 172.21.15.67:0/1505330965 wait complete. 2024-09-10T23:03:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 Processor -- start 2024-09-10T23:03:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 -- start start 2024-09-10T23:03:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab8101fc0 0x7faab819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 0x7faab81a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faab8114c40 con 0x7faab81016a0 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faab8114ac0 con 0x7faab810f250 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.346+0000 7faab7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faab8114dc0 con 0x7faab8101fc0 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47528/0 (socket says 172.21.15.67:47528) 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 -- 172.21.15.67:0/3043932172 learned_addr learned my addr 172.21.15.67:0/3043932172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5400640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab8101fc0 0x7faab819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 -- 172.21.15.67:0/3043932172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab8101fc0 msgr2=0x7faab819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab8101fc0 0x7faab819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 -- 172.21.15.67:0/3043932172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 msgr2=0x7faab81a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab6800640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 0x7faab81a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 0x7faab81a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.347+0000 7faab5e00640 1 -- 172.21.15.67:0/3043932172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab81a1260 con 0x7faab81016a0 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faab6800640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 0x7faab81a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faab5e00640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ea60 tx=0x7faaa800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faaaea00640 1 -- 172.21.15.67:0/3043932172 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800cde0 con 0x7faab81016a0 2024-09-10T23:03:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab81a1550 con 0x7faab81016a0 2024-09-10T23:03:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faaaea00640 1 -- 172.21.15.67:0/3043932172 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaa80035f0 con 0x7faab81016a0 2024-09-10T23:03:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faaaea00640 1 -- 172.21.15.67:0/3043932172 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8010690 con 0x7faab81016a0 2024-09-10T23:03:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.348+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a7de0 con 0x7faab81016a0 2024-09-10T23:03:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.352+0000 7faabfc24740 1 Processor -- start 2024-09-10T23:03:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.352+0000 7faabfc24740 1 -- start start 2024-09-10T23:03:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 0x55cedd2a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a9c10 0x55cedd2b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cedd21eca0 con 0x55cedd2a9c10 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cedd20e0d0 con 0x55cedd2a92f0 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faabfc24740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cedd211ee0 con 0x55cedd2b5ba0 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faaaea00640 1 -- 172.21.15.67:0/3043932172 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faaa8004000 con 0x7faab81016a0 2024-09-10T23:03:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a9c10 0x55cedd2b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faab5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faab5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57670/0 (socket says 172.21.15.67:57670) 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faab5400640 1 -- 172.21.15.67:0/1881726012 learned_addr learned my addr 172.21.15.67:0/1881726012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.353+0000 7faab6800640 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 0x55cedd2a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 -- 172.21.15.67:0/1881726012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 msgr2=0x55cedd2a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 0x55cedd2a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 -- 172.21.15.67:0/1881726012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a9c10 msgr2=0x55cedd2b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a9c10 0x55cedd2b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 -- 172.21.15.67:0/1881726012 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cedd1c0400 con 0x55cedd2b5ba0 2024-09-10T23:03:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab6800640 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 0x55cedd2a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.354+0000 7faab5400640 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000c00 tx=0x7faaa4031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d90bbbf5bea064e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.355+0000 7faaa3400640 1 -- 172.21.15.67:0/1881726012 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4035030 con 0x55cedd2b5ba0 2024-09-10T23:03:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.355+0000 7faaa3400640 1 -- 172.21.15.67:0/1881726012 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaa4040070 con 0x55cedd2b5ba0 2024-09-10T23:03:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.355+0000 7faaa3400640 1 -- 172.21.15.67:0/1881726012 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4003850 con 0x55cedd2b5ba0 2024-09-10T23:03:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.355+0000 7faaaea00640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 0x7faa9c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.355+0000 7faaaea00640 1 -- 172.21.15.67:0/3043932172 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7faaa80e6600 con 0x7faab81016a0 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faab6800640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 0x7faa9c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 -- 172.21.15.67:0/1881726012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 msgr2=0x55cedd2b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000c00 tx=0x7faaa4031ba0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 -- 172.21.15.67:0/1881726012 shutdown_connections 2024-09-10T23:03:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b7f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a9c10 0x55cedd2b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 --2- 172.21.15.67:0/1881726012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a92f0 0x55cedd2a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.356+0000 7faabfc24740 1 -- 172.21.15.67:0/1881726012 >> 172.21.15.67:0/1881726012 conn(0x55cedd2a7120 msgr2=0x55cedd2a82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.357+0000 7faab6800640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 0x7faa9c0b7a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faab0005fd0 tx=0x7faab0009590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.357+0000 7faabfc24740 1 -- 172.21.15.67:0/1881726012 shutdown_connections 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.357+0000 7faabfc24740 1 -- 172.21.15.67:0/1881726012 wait complete. 2024-09-10T23:03:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.357+0000 7faabfc24740 1 -- start start 2024-09-10T23:03:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.358+0000 7faabfc24740 1 Processor -- start 2024-09-10T23:03:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.358+0000 7faabfc24740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.358+0000 7faabfc24740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.358+0000 7faabfc24740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faabfc24740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cedd06f7a0 con 0x55cedd2a92f0 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faabfc24740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cedd1b3a10 con 0x55cedd2b5ba0 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faabfc24740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cedd2b87c0 con 0x55cedd2a9c10 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faabfc24740 10 client.0 _ll_register_callbacks cb 0x55cedd1c3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47556/0 (socket says 172.21.15.67:47556) 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 -- 172.21.15.67:0/1119090387 learned_addr learned my addr 172.21.15.67:0/1119090387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5400640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab6800640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 -- 172.21.15.67:0/1119090387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 msgr2=0x55cedd2b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 -- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 msgr2=0x55cedd2b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5e00640 1 -- 172.21.15.67:0/1119090387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cedcf7bb90 con 0x55cedd2a92f0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab5400640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.359+0000 7faab6800640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.360+0000 7faab5e00640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa80a8650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0af10daa8497e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.360+0000 7faa97e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.360+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80abe60 con 0x55cedd2a92f0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.360+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7faaa8004320 con 0x55cedd2a92f0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa4040070 con 0x55cedd2a92f0 2024-09-10T23:03:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faaa80ac200 con 0x55cedd2a92f0 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faabfc24740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cedd2c40c0 con 0x55cedd2a92f0 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaa4040070 con 0x55cedd2a92f0 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.361+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7faaa80e3c40 con 0x55cedd2a92f0 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 10 client.15807 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:03:29.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 20 client.15807 populate_metadata read hostname 'smithi067' 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd2c5620 0x55cedd2c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 4 client.15807 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cedd220bf0 con 0x55cedd2c5620 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.362+0000 7faab5400640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd2c5620 0x55cedd2c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.363+0000 7faab5400640 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd2c5620 0x55cedd2c7a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf29072ea0c0b51 server_cookie=1f93e261cd97f85f in_seq=0 out_seq=0 2024-09-10T23:03:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.363+0000 7faa97e00640 10 client.15807 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.364+0000 7faa97e00640 1 -- 172.21.15.67:0/1119090387 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cedd220bf0 con 0x55cedd2c5620 2024-09-10T23:03:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.364+0000 7faa97e00640 10 client.15807 handle_command_reply: tid=0 2024-09-10T23:03:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.364+0000 7faabfc24740 1 client.15807 shutdown 2024-09-10T23:03:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 msgr2=0x55cedd2b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa80a8650 comp rx=0 tx=0).stop 2024-09-10T23:03:29.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 shutdown_connections 2024-09-10T23:03:29.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd2c5620 0x55cedd2c7a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 --2- 172.21.15.67:0/1119090387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2b53c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faa97e00640 0 client.15807 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 >> 172.21.15.67:0/1119090387 conn(0x55cedd2a7120 msgr2=0x55cedd2accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 shutdown_connections 2024-09-10T23:03:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.365+0000 7faabfc24740 1 -- 172.21.15.67:0/1119090387 wait complete. 2024-09-10T23:03:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.366+0000 7faabfc24740 20 client.15807 trim_cache size 0 max 16384 2024-09-10T23:03:29.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.392+0000 7faabfc24740 1 Processor -- start 2024-09-10T23:03:29.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.392+0000 7faabfc24740 1 -- start start 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 0x55cedd2b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a9c10 0x55cedd2afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 0x55cedd2aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faab81b46a0 con 0x55cedd2b5ba0 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faab81b1f70 con 0x55cedd2a9c10 2024-09-10T23:03:29.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faabfc24740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faab81afdc0 con 0x55cedd2a92f0 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faab6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 0x55cedd2aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faab6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 0x55cedd2aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47572/0 (socket says 172.21.15.67:47572) 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faab5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 0x55cedd2b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faab6800640 1 -- 172.21.15.67:0/1838299505 learned_addr learned my addr 172.21.15.67:0/1838299505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.393+0000 7faab5400640 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a9c10 0x55cedd2afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 -- 172.21.15.67:0/1838299505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a9c10 msgr2=0x55cedd2afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a9c10 0x55cedd2afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 -- 172.21.15.67:0/1838299505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 msgr2=0x55cedd2aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 0x55cedd2aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 -- 172.21.15.67:0/1838299505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cedd2a7ba0 con 0x55cedd2a92f0 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faab5e00640 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 0x55cedd2b20a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaa80a9960 tx=0x7faaa8003a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491aa7df459cf50b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faaa2a00640 1 -- 172.21.15.67:0/1838299505 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80abe60 con 0x55cedd2a92f0 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faaa2a00640 1 -- 172.21.15.67:0/1838299505 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaa80acdf0 con 0x55cedd2a92f0 2024-09-10T23:03:29.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.394+0000 7faaa2a00640 1 -- 172.21.15.67:0/1838299505 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80a8d40 con 0x55cedd2a92f0 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 -- 172.21.15.67:0/1838299505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 msgr2=0x55cedd2b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 0x55cedd2b20a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaa80a9960 tx=0x7faaa8003a30 comp rx=0 tx=0).stop 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 -- 172.21.15.67:0/1838299505 shutdown_connections 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2b5ba0 0x55cedd2aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2a9c10 0x55cedd2afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 --2- 172.21.15.67:0/1838299505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a92f0 0x55cedd2b20a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.395+0000 7faabfc24740 1 -- 172.21.15.67:0/1838299505 >> 172.21.15.67:0/1838299505 conn(0x55cedd2a7120 msgr2=0x55cedd2b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.396+0000 7faabfc24740 1 -- 172.21.15.67:0/1838299505 shutdown_connections 2024-09-10T23:03:29.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.396+0000 7faabfc24740 1 -- 172.21.15.67:0/1838299505 wait complete. 2024-09-10T23:03:29.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.396+0000 7faabfc24740 1 -- start start 2024-09-10T23:03:29.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.396+0000 7faabfc24740 1 Processor -- start 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faab81b0430 con 0x55cedd2a92f0 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faab81b18a0 con 0x55cedd2b5ba0 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faab81b0b00 con 0x55cedd2a9c10 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faabfc24740 10 client.0 _ll_register_callbacks cb 0x7faaa800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faab5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47586/0 (socket says 172.21.15.67:47586) 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.397+0000 7faab5e00640 1 -- 172.21.15.67:0/3220536415 learned_addr learned my addr 172.21.15.67:0/3220536415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab6800640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5400640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5e00640 1 -- 172.21.15.67:0/3220536415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 msgr2=0x55cedd2ba950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5e00640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2ba950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5e00640 1 -- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 msgr2=0x55cedd2bb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5e00640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2bb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5400640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2ba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:29.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.398+0000 7faab5e00640 1 -- 172.21.15.67:0/3220536415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cedd332510 con 0x55cedd2a92f0 2024-09-10T23:03:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faab5e00640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d010 tx=0x7faaa80ec550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3f326da1134cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faa97e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:03:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80ac6f0 con 0x55cedd2a92f0 2024-09-10T23:03:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7faaa80a8960 con 0x55cedd2a92f0 2024-09-10T23:03:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa80acdf0 con 0x55cedd2a92f0 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.399+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faaa80e6240 con 0x55cedd2a92f0 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 10 client.15813 fetch_fsmap learned FSMap version 30 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cedd332510 con 0x55cedd2a92f0 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaa80acdf0 con 0x55cedd2a92f0 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7faaa80ab3c0 con 0x55cedd2a92f0 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 10 client.15813 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 10 client.15813 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 20 client.15813 populate_metadata read hostname 'smithi067' 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd333b10 0x55cedd335f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 4 client.15813 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:03:29.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.400+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55cedd21c6a0 con 0x55cedd333b10 2024-09-10T23:03:29.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.401+0000 7faab5400640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd333b10 0x55cedd335f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:29.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.402+0000 7faab5400640 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd333b10 0x55cedd335f00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bd64895d33805a server_cookie=bcbacd54a514a85a in_seq=0 out_seq=0 2024-09-10T23:03:29.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.402+0000 7faa97e00640 10 client.15813 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:29.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faa97e00640 1 -- 172.21.15.67:0/3220536415 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cedd21c6a0 con 0x55cedd333b10 2024-09-10T23:03:29.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faa97e00640 10 client.15813 handle_command_reply: tid=0 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faabfc24740 1 client.15813 shutdown 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 msgr2=0x55cedd2be470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d010 tx=0x7faaa80ec550 comp rx=0 tx=0).stop 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 shutdown_connections 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.516+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cedd333b10 0x55cedd335f00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cedd2b5ba0 0x55cedd2bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cedd2a9c10 0x55cedd2ba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 --2- 172.21.15.67:0/3220536415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cedd2a92f0 0x55cedd2be470 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 >> 172.21.15.67:0/3220536415 conn(0x55cedd2a7120 msgr2=0x55cedd2a5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faa97e00640 0 client.15813 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 shutdown_connections 2024-09-10T23:03:29.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 1 -- 172.21.15.67:0/3220536415 wait complete. 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.517+0000 7faabfc24740 20 client.15813 trim_cache size 0 max 16384 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 30, 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 14, 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "95d0cd50": { 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 14, 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:03:29.522 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:29.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 msgr2=0x7faa9c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 0x7faa9c0b7a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faab0005fd0 tx=0x7faab0009590 comp rx=0 tx=0).stop 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 msgr2=0x7faab819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:29.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ea60 tx=0x7faaa800ef20 comp rx=0 tx=0).stop 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 shutdown_connections 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faa9c0b5580 0x7faa9c0b7a40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faab810f250 0x7faab81a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab8101fc0 0x7faab819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 --2- 172.21.15.67:0/3043932172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faab81016a0 0x7faab819b1b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 >> 172.21.15.67:0/3043932172 conn(0x7faab80fd5b0 msgr2=0x7faab8110580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 shutdown_connections 2024-09-10T23:03:29.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:29.518+0000 7faab7200640 1 -- 172.21.15.67:0/3043932172 wait complete. 2024-09-10T23:03:29.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:29 smithi145 ceph-mon[34397]: pgmap v525: 129 pgs: 129 active+clean; 1.4 GiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:03:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:29 smithi092 ceph-mon[34223]: pgmap v525: 129 pgs: 129 active+clean; 1.4 GiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:03:30.457 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '95d0cd50', quiesce took 4.7 seconds. Will release after: 25.3 2024-09-10T23:03:31.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:31 smithi067 ceph-mon[30939]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:03:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:31 smithi145 ceph-mon[34397]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:03:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:31 smithi092 ceph-mon[34223]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:03:33.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:33 smithi067 ceph-mon[30939]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:03:33.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:33 smithi145 ceph-mon[34397]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:03:33.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:33 smithi092 ceph-mon[34223]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:03:35.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:35 smithi145 ceph-mon[34397]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:03:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:35 smithi092 ceph-mon[34223]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:03:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:35 smithi067 ceph-mon[30939]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:03:37.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:37 smithi145 ceph-mon[34397]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:03:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:37 smithi092 ceph-mon[34223]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:03:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:37 smithi067 ceph-mon[30939]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:03:39.600 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:39 smithi067 ceph-mon[30939]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:03:39.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:39 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:39 smithi145 ceph-mon[34397]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:03:39.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:39 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:39 smithi092 ceph-mon[34223]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:03:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:39 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:40.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:40.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:40.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:03:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:40.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:03:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:03:41.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:41 smithi145 ceph-mon[34397]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 10 op/s 2024-09-10T23:03:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:41 smithi092 ceph-mon[34223]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 10 op/s 2024-09-10T23:03:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:41 smithi067 ceph-mon[30939]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 10 op/s 2024-09-10T23:03:43.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:43 smithi145 ceph-mon[34397]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:03:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:43 smithi092 ceph-mon[34223]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:03:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:43 smithi067 ceph-mon[30939]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:03:44.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:44 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:44.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:44 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:44.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:44 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:03:45.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:45 smithi145 ceph-mon[34397]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 7 op/s 2024-09-10T23:03:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:45 smithi092 ceph-mon[34223]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 7 op/s 2024-09-10T23:03:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:45 smithi067 ceph-mon[30939]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 7 op/s 2024-09-10T23:03:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:47 smithi145 ceph-mon[34397]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:47.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:47 smithi092 ceph-mon[34223]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:47 smithi067 ceph-mon[30939]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:49 smithi145 ceph-mon[34397]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 546 KiB/s wr, 5 op/s 2024-09-10T23:03:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:49 smithi092 ceph-mon[34223]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 546 KiB/s wr, 5 op/s 2024-09-10T23:03:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:49 smithi067 ceph-mon[30939]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 546 KiB/s wr, 5 op/s 2024-09-10T23:03:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:51 smithi145 ceph-mon[34397]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:51 smithi092 ceph-mon[34223]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:51 smithi067 ceph-mon[30939]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 546 KiB/s wr, 7 op/s 2024-09-10T23:03:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:53 smithi145 ceph-mon[34397]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:53 smithi092 ceph-mon[34223]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:53 smithi067 ceph-mon[30939]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:55 smithi145 ceph-mon[34397]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:55.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:55 smithi092 ceph-mon[34223]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:55 smithi067 ceph-mon[30939]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T23:03:56.149 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.745+0000 7f2537800640 1 -- 172.21.15.67:0/1339161973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530102da0 msgr2=0x7f2530103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.745+0000 7f2537800640 1 --2- 172.21.15.67:0/1339161973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530102da0 0x7f2530103180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2520009900 tx=0x7f252002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 -- 172.21.15.67:0/1339161973 shutdown_connections 2024-09-10T23:03:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 --2- 172.21.15.67:0/1339161973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530109130 0x7f253010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 --2- 172.21.15.67:0/1339161973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 0x7f2530108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 --2- 172.21.15.67:0/1339161973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530102da0 0x7f2530103180 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 -- 172.21.15.67:0/1339161973 >> 172.21.15.67:0/1339161973 conn(0x7f25300fe810 msgr2=0x7f2530100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 -- 172.21.15.67:0/1339161973 shutdown_connections 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.746+0000 7f2537800640 1 -- 172.21.15.67:0/1339161973 wait complete. 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 Processor -- start 2024-09-10T23:03:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 -- start start 2024-09-10T23:03:56.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 0x7f253019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:56.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 0x7f253019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2530112cb0 con 0x7f2530109130 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2530112b30 con 0x7f25301087b0 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2537800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2530112e30 con 0x7f2530102da0 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.747+0000 7f2535800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47168/0 (socket says 172.21.15.67:47168) 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 -- 172.21.15.67:0/3893786043 learned_addr learned my addr 172.21.15.67:0/3893786043 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f252fe00640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 0x7f253019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2534e00640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 0x7f253019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 -- 172.21.15.67:0/3893786043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 msgr2=0x7f253019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 0x7f253019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 -- 172.21.15.67:0/3893786043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 msgr2=0x7f253019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 0x7f253019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 -- 172.21.15.67:0/3893786043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25301a14b0 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2534e00640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 0x7f253019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.748+0000 7f2535800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f252800eaa0 tx=0x7f252800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.749+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252800cdb0 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.749+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2528005020 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.749+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25301a17a0 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.749+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25280116d0 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.749+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25301a7e60 con 0x7f2530109130 2024-09-10T23:03:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.750+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530103180 con 0x7f2530109130 2024-09-10T23:03:56.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.752+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f252800f040 con 0x7f2530109130 2024-09-10T23:03:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.753+0000 7f252d600640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 0x7f25140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.754+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f25280e6230 con 0x7f2530109130 2024-09-10T23:03:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.754+0000 7f2534e00640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 0x7f25140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.755+0000 7f2534e00640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 0x7f25140b7910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2520006010 tx=0x7f2520005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:56.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.758+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2528016030 con 0x7f2530109130 2024-09-10T23:03:56.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.989+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f253002e5c0 con 0x7f2530109130 2024-09-10T23:03:56.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.991+0000 7f252d600640 1 -- 172.21.15.67:0/3893786043 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f25280ac9c0 con 0x7f2530109130 2024-09-10T23:03:56.993 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:03:56.994 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:03:56.994 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:03:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 msgr2=0x7f25140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 0x7f25140b7910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2520006010 tx=0x7f2520005a20 comp rx=0 tx=0).stop 2024-09-10T23:03:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 msgr2=0x7f25301a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f252800eaa0 tx=0x7f252800ef60 comp rx=0 tx=0).stop 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 shutdown_connections 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f25140b5450 0x7f25140b7910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2530109130 0x7f25301a0ce0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25301087b0 0x7f253019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 --2- 172.21.15.67:0/3893786043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2530102da0 0x7f253019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 >> 172.21.15.67:0/3893786043 conn(0x7f25300fe810 msgr2=0x7f2530075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 shutdown_connections 2024-09-10T23:03:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:56.993+0000 7f2537800640 1 -- 172.21.15.67:0/3893786043 wait complete. 2024-09-10T23:03:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:57 smithi145 ceph-mon[34397]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:03:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:57 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3893786043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:57 smithi092 ceph-mon[34223]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:03:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:57 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3893786043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:57 smithi067 ceph-mon[30939]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:03:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:57 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3893786043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:57.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 95d0cd50 --release --await' 2024-09-10T23:03:57.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 95d0cd50 --release --await 2024-09-10T23:03:58.265 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 -- 172.21.15.67:0/1732484602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108620 msgr2=0x7f5664108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 --2- 172.21.15.67:0/1732484602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108620 0x7f5664108a60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 -- 172.21.15.67:0/1732484602 shutdown_connections 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 --2- 172.21.15.67:0/1732484602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664108fa0 0x7f566410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 --2- 172.21.15.67:0/1732484602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108620 0x7f5664108a60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 --2- 172.21.15.67:0/1732484602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664102c10 0x7f5664102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 -- 172.21.15.67:0/1732484602 >> 172.21.15.67:0/1732484602 conn(0x7f56640fe6c0 msgr2=0x7f5664100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 -- 172.21.15.67:0/1732484602 shutdown_connections 2024-09-10T23:03:58.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.855+0000 7f5668a00640 1 -- 172.21.15.67:0/1732484602 wait complete. 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 Processor -- start 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 -- start start 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664102c10 0x7f566419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664108620 0x7f566419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5664112b20 con 0x7f5664108fa0 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56641129a0 con 0x7f5664108620 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.856+0000 7f5668a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5664112ca0 con 0x7f5664102c10 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45104/0 (socket says 172.21.15.67:45104) 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 -- 172.21.15.67:0/2821083550 learned_addr learned my addr 172.21.15.67:0/2821083550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5662a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664108620 0x7f566419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663400640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664102c10 0x7f566419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 -- 172.21.15.67:0/2821083550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664102c10 msgr2=0x7f566419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664102c10 0x7f566419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 -- 172.21.15.67:0/2821083550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664108620 msgr2=0x7f566419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664108620 0x7f566419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.857+0000 7f5663e00640 1 -- 172.21.15.67:0/2821083550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641a1290 con 0x7f5664108fa0 2024-09-10T23:03:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f5663e00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f565c00eaa0 tx=0x7f565c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f565be00640 1 -- 172.21.15.67:0/2821083550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00cdb0 con 0x7f5664108fa0 2024-09-10T23:03:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f565be00640 1 -- 172.21.15.67:0/2821083550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565c005020 con 0x7f5664108fa0 2024-09-10T23:03:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f565be00640 1 -- 172.21.15.67:0/2821083550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c0116d0 con 0x7f5664108fa0 2024-09-10T23:03:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56641a1580 con 0x7f5664108fa0 2024-09-10T23:03:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.858+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641a7e10 con 0x7f5664108fa0 2024-09-10T23:03:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.861+0000 7f566ca7a740 1 Processor -- start 2024-09-10T23:03:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.861+0000 7f565be00640 1 -- 172.21.15.67:0/2821083550 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f565c00f040 con 0x7f5664108fa0 2024-09-10T23:03:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 -- start start 2024-09-10T23:03:58.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f5d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559773ec8290 con 0x559773f511d0 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559773eb76c0 con 0x559773f51af0 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f566ca7a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559773ebb4d0 con 0x559773f5da80 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.862+0000 7f5662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45118/0 (socket says 172.21.15.67:45118) 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 -- 172.21.15.67:0/2434104506 learned_addr learned my addr 172.21.15.67:0/2434104506 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5663400640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f5d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 -- 172.21.15.67:0/2434104506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 msgr2=0x559773f5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 -- 172.21.15.67:0/2434104506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 msgr2=0x559773f5d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f5d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 -- 172.21.15.67:0/2434104506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559773e44460 con 0x559773f511d0 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5663400640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.863+0000 7f5662a00640 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5650033000 tx=0x7f5650000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7e1334fe4911a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f564fe00640 1 -- 172.21.15.67:0/2434104506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650035040 con 0x559773f511d0 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f565be00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 0x7f56480b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f564fe00640 1 -- 172.21.15.67:0/2434104506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5650045070 con 0x559773f511d0 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f564fe00640 1 -- 172.21.15.67:0/2434104506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650003670 con 0x559773f511d0 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f565be00640 1 -- 172.21.15.67:0/2821083550 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f565c0e6150 con 0x7f5664108fa0 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.864+0000 7f5663400640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 0x7f56480b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- 172.21.15.67:0/2434104506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 msgr2=0x559773f515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5650033000 tx=0x7f5650000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- 172.21.15.67:0/2434104506 shutdown_connections 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 --2- 172.21.15.67:0/2434104506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f515b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- 172.21.15.67:0/2434104506 >> 172.21.15.67:0/2434104506 conn(0x559773f4f000 msgr2=0x559773f501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- 172.21.15.67:0/2434104506 shutdown_connections 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f5663400640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 0x7f56480b7a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x559773f54350 tx=0x7f565400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- 172.21.15.67:0/2434104506 wait complete. 2024-09-10T23:03:58.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.865+0000 7f566ca7a740 1 -- start start 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 Processor -- start 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559773d18bb0 con 0x559773f51af0 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559773e526c0 con 0x559773f511d0 2024-09-10T23:03:58.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559773f606d0 con 0x559773f5da80 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f566ca7a740 10 client.0 _ll_register_callbacks cb 0x559773e6d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f5662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f5662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45132/0 (socket says 172.21.15.67:45132) 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.866+0000 7f5662a00640 1 -- 172.21.15.67:0/3199945849 learned_addr learned my addr 172.21.15.67:0/3199945849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5663e00640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5663400640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 -- 172.21.15.67:0/3199945849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 msgr2=0x559773f5b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 -- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 msgr2=0x559773f5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 -- 172.21.15.67:0/3199945849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559773c15b00 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f5662a00640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56500049b0 tx=0x7f56500049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821a38d306c93c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f564cc00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650035030 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f56500051e0 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.867+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5650045070 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5650057050 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 10 client.15831 fetch_fsmap learned FSMap version 31 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559773c8c960 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5650045070 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5650003c20 con 0x559773f51af0 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 10 client.15831 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 10 client.15831 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:03:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 20 client.15831 populate_metadata read hostname 'smithi067' 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773f6d4f0 0x559773f6f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 4 client.15831 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.868+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559773eca1e0 con 0x559773f6d4f0 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.869+0000 7f5663400640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773f6d4f0 0x559773f6f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.869+0000 7f5663400640 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773f6d4f0 0x559773f6f8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3228d6816595cee server_cookie=7589c85dfb5dc5fb in_seq=0 out_seq=0 2024-09-10T23:03:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.870+0000 7f564cc00640 10 client.15831 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.870+0000 7f564cc00640 1 -- 172.21.15.67:0/3199945849 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559773eca1e0 con 0x559773f6d4f0 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.870+0000 7f564cc00640 10 client.15831 handle_command_reply: tid=0 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.870+0000 7f566ca7a740 1 client.15831 shutdown 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 msgr2=0x559773f5b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56500049b0 tx=0x7f56500049e0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 shutdown_connections 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773f6d4f0 0x559773f6f8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f5b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f51af0 0x559773f5b010 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3199945849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 >> 172.21.15.67:0/3199945849 conn(0x559773f4f000 msgr2=0x559773f54ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f564cc00640 0 client.15831 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:58.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 shutdown_connections 2024-09-10T23:03:58.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 1 -- 172.21.15.67:0/3199945849 wait complete. 2024-09-10T23:03:58.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.871+0000 7f566ca7a740 20 client.15831 trim_cache size 0 max 16384 2024-09-10T23:03:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.889+0000 7f566ca7a740 1 Processor -- start 2024-09-10T23:03:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.889+0000 7f566ca7a740 1 -- start start 2024-09-10T23:03:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.889+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.889+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f57ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.889+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f58220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f566ca7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b45a0 con 0x559773f511d0 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f566ca7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b1e70 con 0x559773f51af0 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f566ca7a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f56641afcc0 con 0x559773f5da80 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45134/0 (socket says 172.21.15.67:45134) 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 -- 172.21.15.67:0/419750531 learned_addr learned my addr 172.21.15.67:0/419750531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663e00640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f58220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5662a00640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f57ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 -- 172.21.15.67:0/419750531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 msgr2=0x559773f58220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f58220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 -- 172.21.15.67:0/419750531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 msgr2=0x559773f57ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f57ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5663400640 1 -- 172.21.15.67:0/419750531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559773f4fa80 con 0x559773f511d0 2024-09-10T23:03:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.890+0000 7f5662a00640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f57ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f5663400640 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f565400b980 tx=0x7f5654016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8c3088115e4f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f564f400640 1 -- 172.21.15.67:0/419750531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654003d80 con 0x559773f511d0 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f564f400640 1 -- 172.21.15.67:0/419750531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56540200d0 con 0x559773f511d0 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f564f400640 1 -- 172.21.15.67:0/419750531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654029d50 con 0x559773f511d0 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f566ca7a740 1 -- 172.21.15.67:0/419750531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 msgr2=0x559773f506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f566ca7a740 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f565400b980 tx=0x7f5654016000 comp rx=0 tx=0).stop 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.891+0000 7f566ca7a740 1 -- 172.21.15.67:0/419750531 shutdown_connections 2024-09-10T23:03:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f5da80 0x559773f58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f51af0 0x559773f57ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 --2- 172.21.15.67:0/419750531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f511d0 0x559773f506c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 -- 172.21.15.67:0/419750531 >> 172.21.15.67:0/419750531 conn(0x559773f4f000 msgr2=0x559773f5c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 -- 172.21.15.67:0/419750531 shutdown_connections 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 -- 172.21.15.67:0/419750531 wait complete. 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 -- start start 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.892+0000 7f566ca7a740 1 Processor -- start 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f51af0 0x559773f62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b0330 con 0x559773f5da80 2024-09-10T23:03:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b17a0 con 0x559773f511d0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f56641b0a00 con 0x559773f51af0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f566ca7a740 10 client.0 _ll_register_callbacks cb 0x7f565400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f5663e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f5663e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45138/0 (socket says 172.21.15.67:45138) 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f5663e00640 1 -- 172.21.15.67:0/3520405972 learned_addr learned my addr 172.21.15.67:0/3520405972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f5662a00640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f51af0 0x559773f62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.893+0000 7f5663400640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 -- 172.21.15.67:0/3520405972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f51af0 msgr2=0x559773f62690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f51af0 0x559773f62690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 -- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 msgr2=0x559773f661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 -- 172.21.15.67:0/3520405972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559773f63440 con 0x559773f5da80 2024-09-10T23:03:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663400640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f5663e00640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f565c0abf90 tx=0x7f565c0aba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5995e08c173beef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f564cc00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01f040 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56540200d0 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f565c0cdbc0 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f565c0ac290 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f566ca7a740 10 client.15837 fetch_fsmap learned FSMap version 31 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.894+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559773fdb1a0 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56540200d0 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f565c0a7be0 con 0x559773f5da80 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 10 client.15837 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 10 client.15837 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 20 client.15837 populate_metadata read hostname 'smithi067' 2024-09-10T23:03:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773fdb8c0 0x559773fddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 4 client.15837 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "95d0cd50", "release": true, "await": true}] 2024-09-10T23:03:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "95d0cd50", "release": true, "await": true}) -- 0x559773ec5c90 con 0x559773fdb8c0 2024-09-10T23:03:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.895+0000 7f5663400640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773fdb8c0 0x559773fddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.896+0000 7f5663400640 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773fdb8c0 0x559773fddcb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c153dd964ca52627 server_cookie=bd0a85d30788fa7 in_seq=0 out_seq=0 2024-09-10T23:03:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.896+0000 7f564cc00640 10 client.15837 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:58.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f564cc00640 1 -- 172.21.15.67:0/3520405972 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559773ec5c90 con 0x559773fdb8c0 2024-09-10T23:03:58.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f564cc00640 10 client.15837 handle_command_reply: tid=0 2024-09-10T23:03:58.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 client.15837 shutdown 2024-09-10T23:03:58.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 msgr2=0x559773f62c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f565c0abf90 tx=0x7f565c0aba00 comp rx=0 tx=0).stop 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 shutdown_connections 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559773fdb8c0 0x559773fddcb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559773f5da80 0x559773f62c10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559773f51af0 0x559773f62690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 --2- 172.21.15.67:0/3520405972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559773f511d0 0x559773f661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 >> 172.21.15.67:0/3520405972 conn(0x559773f4f000 msgr2=0x559773f68950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f564cc00640 0 client.15837 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 shutdown_connections 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.924+0000 7f566ca7a740 1 -- 172.21.15.67:0/3520405972 wait complete. 2024-09-10T23:03:58.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f566ca7a740 20 client.15837 trim_cache size 0 max 16384 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 16, 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "95d0cd50": { 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 16, 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:03:58.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:58.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:58.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:03:58.929 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:03:58.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 msgr2=0x7f56480b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 0x7f56480b7a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x559773f54350 tx=0x7f565400b390 comp rx=0 tx=0).stop 2024-09-10T23:03:58.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 msgr2=0x7f56641a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f565c00eaa0 tx=0x7f565c00ef60 comp rx=0 tx=0).stop 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 shutdown_connections 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f56480b5580 0x7f56480b7a40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5664108fa0 0x7f56641a0ac0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5664108620 0x7f566419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 --2- 172.21.15.67:0/2821083550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5664102c10 0x7f566419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.925+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 >> 172.21.15.67:0/2821083550 conn(0x7f56640fe6c0 msgr2=0x7f5664105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.926+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 shutdown_connections 2024-09-10T23:03:58.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:03:58.926+0000 7f5668a00640 1 -- 172.21.15.67:0/2821083550 wait complete. 2024-09-10T23:03:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:03:59 smithi145 ceph-mon[34397]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:03:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:03:59 smithi092 ceph-mon[34223]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:03:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:03:59 smithi067 ceph-mon[30939]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:03:59.992 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '95d0cd50', total seconds elapsed: 34.3 2024-09-10T23:03:59.992 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:03:59.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:04:00.454 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 -- 172.21.15.67:0/73031265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca8104e10 msgr2=0x7faca81051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 --2- 172.21.15.67:0/73031265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca8104e10 0x7faca81051f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 -- 172.21.15.67:0/73031265 shutdown_connections 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 --2- 172.21.15.67:0/73031265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca81119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 --2- 172.21.15.67:0/73031265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca810a820 0x7faca810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 --2- 172.21.15.67:0/73031265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca8104e10 0x7faca81051f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 -- 172.21.15.67:0/73031265 >> 172.21.15.67:0/73031265 conn(0x7faca8100840 msgr2=0x7faca8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 -- 172.21.15.67:0/73031265 shutdown_connections 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.089+0000 7facaee00640 1 -- 172.21.15.67:0/73031265 wait complete. 2024-09-10T23:04:01.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.090+0000 7facaee00640 1 Processor -- start 2024-09-10T23:04:01.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.090+0000 7facaee00640 1 -- start start 2024-09-10T23:04:01.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:01.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 0x7faca810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:01.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:01.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faca8114d60 con 0x7faca8104e10 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faca8114be0 con 0x7faca810b1a0 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facaee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faca8114ee0 con 0x7faca810a820 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facada00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facad000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 0x7faca810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facada00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45162/0 (socket says 172.21.15.67:45162) 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facad000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 0x7faca810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:45944/0 (socket says 172.21.15.67:45944) 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facada00640 1 -- 172.21.15.67:0/1546591859 learned_addr learned my addr 172.21.15.67:0/1546591859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:01.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.091+0000 7facae400640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 -- 172.21.15.67:0/1546591859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 msgr2=0x7faca810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 0x7faca810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 -- 172.21.15.67:0/1546591859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 msgr2=0x7faca810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 -- 172.21.15.67:0/1546591859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca810f6d0 con 0x7faca8104e10 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facae400640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:01.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.092+0000 7facada00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac9c02a910 tx=0x7fac9c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.093+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03f070 con 0x7faca8104e10 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.093+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac9c003e40 con 0x7faca8104e10 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.093+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81abaa0 con 0x7faca8104e10 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.093+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c035030 con 0x7faca8104e10 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.093+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81abfa0 con 0x7faca8104e10 2024-09-10T23:04:01.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.094+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca81051f0 con 0x7faca8104e10 2024-09-10T23:04:01.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.097+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fac9c031400 con 0x7faca8104e10 2024-09-10T23:04:01.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.099+0000 7faca6000640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 0x7fac900b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:01.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.099+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fac9c1071f0 con 0x7faca8104e10 2024-09-10T23:04:01.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.099+0000 7facad000640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 0x7fac900b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:01.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.100+0000 7facad000640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 0x7fac900b7a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faca8111420 tx=0x7fac9400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:01.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.101+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac9c0cd980 con 0x7faca8104e10 2024-09-10T23:04:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.391+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca802e5c0 con 0x7faca8104e10 2024-09-10T23:04:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.392+0000 7faca6000640 1 -- 172.21.15.67:0/1546591859 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fac9c0cd280 con 0x7faca8104e10 2024-09-10T23:04:01.393 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:04:01.393 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:04:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:04:01.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 msgr2=0x7fac900b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:01.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 0x7fac900b7a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faca8111420 tx=0x7fac9400b390 comp rx=0 tx=0).stop 2024-09-10T23:04:01.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 msgr2=0x7faca81116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:01.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac9c02a910 tx=0x7fac9c003c20 comp rx=0 tx=0).stop 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 shutdown_connections 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fac900b5580 0x7fac900b7a40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca810b1a0 0x7faca810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.394+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faca810a820 0x7faca810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.395+0000 7facaee00640 1 --2- 172.21.15.67:0/1546591859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca8104e10 0x7faca81116d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.395+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 >> 172.21.15.67:0/1546591859 conn(0x7faca8100840 msgr2=0x7faca8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.395+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 shutdown_connections 2024-09-10T23:04:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:01.395+0000 7facaee00640 1 -- 172.21.15.67:0/1546591859 wait complete. 2024-09-10T23:04:01.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:01 smithi145 ceph-mon[34397]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:01 smithi092 ceph-mon[34223]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:01.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:01 smithi067 ceph-mon[30939]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:02.316 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:04:02.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:04:02.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:02 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1546591859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:02.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:02 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1546591859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:02.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:02 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1546591859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.417+0000 7f4d83200640 1 -- 172.21.15.67:0/2016614432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84075e00 msgr2=0x7f4d8410ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.417+0000 7f4d83200640 1 --2- 172.21.15.67:0/2016614432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84075e00 0x7f4d8410ecc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009900 tx=0x7f4d7002a410 comp rx=0 tx=0).stop 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 -- 172.21.15.67:0/2016614432 shutdown_connections 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 --2- 172.21.15.67:0/2016614432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 --2- 172.21.15.67:0/2016614432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d84075e00 0x7f4d8410ecc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 --2- 172.21.15.67:0/2016614432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d840754e0 0x7f4d840758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 -- 172.21.15.67:0/2016614432 >> 172.21.15.67:0/2016614432 conn(0x7f4d840fe660 msgr2=0x7f4d84100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 -- 172.21.15.67:0/2016614432 shutdown_connections 2024-09-10T23:04:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.418+0000 7f4d83200640 1 -- 172.21.15.67:0/2016614432 wait complete. 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.419+0000 7f4d83200640 1 Processor -- start 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.419+0000 7f4d83200640 1 -- start start 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.419+0000 7f4d83200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.419+0000 7f4d83200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d84075e00 0x7f4d8419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.419+0000 7f4d83200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d83200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114ce0 con 0x7f4d840754e0 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d83200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114b60 con 0x7f4d84075e00 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d83200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114e60 con 0x7f4d8410f200 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45182/0 (socket says 172.21.15.67:45182) 2024-09-10T23:04:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 -- 172.21.15.67:0/741476007 learned_addr learned my addr 172.21.15.67:0/741476007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81400640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d84075e00 0x7f4d8419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d82800640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 -- 172.21.15.67:0/741476007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 msgr2=0x7f4d841a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 -- 172.21.15.67:0/741476007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d84075e00 msgr2=0x7f4d8419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d84075e00 0x7f4d8419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.420+0000 7f4d81e00640 1 -- 172.21.15.67:0/741476007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d841a1240 con 0x7f4d840754e0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d82800640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d81e00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d950 tx=0x7f4d7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d7aa00640 1 -- 172.21.15.67:0/741476007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74017030 con 0x7f4d840754e0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d7aa00640 1 -- 172.21.15.67:0/741476007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d74004020 con 0x7f4d840754e0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841a1530 con 0x7f4d840754e0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d7aa00640 1 -- 172.21.15.67:0/741476007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740055d0 con 0x7f4d840754e0 2024-09-10T23:04:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.421+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a7dc0 con 0x7f4d840754e0 2024-09-10T23:04:03.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 Processor -- start 2024-09-10T23:04:03.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 -- start start 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987809660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x56498780bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564987772ca0 con 0x5649877fdc10 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5649877620d0 con 0x5649877fd2f0 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.425+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564987765ee0 con 0x564987809ba0 2024-09-10T23:04:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d7aa00640 1 -- 172.21.15.67:0/741476007 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4d74009a30 con 0x7f4d840754e0 2024-09-10T23:04:03.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:42222/0 (socket says 172.21.15.67:42222) 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 -- 172.21.15.67:0/2497998939 learned_addr learned my addr 172.21.15.67:0/2497998939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d81e00640 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987809660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d81400640 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x56498780bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 -- 172.21.15.67:0/2497998939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 msgr2=0x56498780bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x56498780bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 -- 172.21.15.67:0/2497998939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 msgr2=0x564987809660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987809660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.426+0000 7f4d82800640 1 -- 172.21.15.67:0/2497998939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564987714400 con 0x5649877fd2f0 2024-09-10T23:04:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.427+0000 7f4d82800640 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009fc0 tx=0x7f4d7c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5db25c7b5b9405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.427+0000 7f4d6f000640 1 -- 172.21.15.67:0/2497998939 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c011700 con 0x5649877fd2f0 2024-09-10T23:04:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.427+0000 7f4d6f000640 1 -- 172.21.15.67:0/2497998939 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d7c011d30 con 0x5649877fd2f0 2024-09-10T23:04:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.427+0000 7f4d6f000640 1 -- 172.21.15.67:0/2497998939 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c01cda0 con 0x5649877fd2f0 2024-09-10T23:04:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.427+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 0x7f4d680b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d7aa00640 1 -- 172.21.15.67:0/741476007 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f4d740e5e00 con 0x7f4d840754e0 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d81e00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 0x7f4d680b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 -- 172.21.15.67:0/2497998939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 msgr2=0x5649877fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009fc0 tx=0x7f4d7c010860 comp rx=0 tx=0).stop 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 -- 172.21.15.67:0/2497998939 shutdown_connections 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x56498780bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987809660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/2497998939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fd2f0 0x5649877fd6d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.428+0000 7f4d8b68c740 1 -- 172.21.15.67:0/2497998939 >> 172.21.15.67:0/2497998939 conn(0x5649877fb120 msgr2=0x5649877fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.429+0000 7f4d8b68c740 1 -- 172.21.15.67:0/2497998939 shutdown_connections 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.429+0000 7f4d8b68c740 1 -- 172.21.15.67:0/2497998939 wait complete. 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.429+0000 7f4d81e00640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 0x7f4d680b7a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5649877fc000 tx=0x7f4d74000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.429+0000 7f4d8b68c740 1 -- start start 2024-09-10T23:04:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.429+0000 7f4d8b68c740 1 Processor -- start 2024-09-10T23:04:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x5649878068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x564987806ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5649875c37a0 con 0x5649877fd2f0 2024-09-10T23:04:03.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564987707a10 con 0x5649877fdc10 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56498780c820 con 0x564987809ba0 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d8b68c740 10 client.0 _ll_register_callbacks cb 0x564987717be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45198/0 (socket says 172.21.15.67:45198) 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.430+0000 7f4d82800640 1 -- 172.21.15.67:0/1743469652 learned_addr learned my addr 172.21.15.67:0/1743469652 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d81400640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x5649878068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x564987806ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 -- 172.21.15.67:0/1743469652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 msgr2=0x564987806ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x564987806ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 -- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 msgr2=0x5649878068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x5649878068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 -- 172.21.15.67:0/1743469652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649874cfb90 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d82800640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009f90 tx=0x7f4d7c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc6c9918a9d1e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d63400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.431+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c024040 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c011d30 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f4d7c004c40 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c004e60 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d8b68c740 10 client.15855 fetch_fsmap learned FSMap version 31 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564987820970 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.432+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d7c011d30 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4d7c0053b0 con 0x5649877fd2f0 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 10 client.15855 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 10 client.15855 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 20 client.15855 populate_metadata read hostname 'smithi067' 2024-09-10T23:04:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564987821d70 0x564987824160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 4 client.15855 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564987774bf0 con 0x564987821d70 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.433+0000 7f4d81400640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564987821d70 0x564987824160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.434+0000 7f4d81400640 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564987821d70 0x564987824160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165a16a80fa2dd24 server_cookie=70d3b71af3e77ac8 in_seq=0 out_seq=0 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.434+0000 7f4d63400640 10 client.15855 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d63400640 1 -- 172.21.15.67:0/1743469652 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564987774bf0 con 0x564987821d70 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d63400640 10 client.15855 handle_command_reply: tid=0 2024-09-10T23:04:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 client.15855 shutdown 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 msgr2=0x564987809470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009f90 tx=0x7f4d7c009900 comp rx=0 tx=0).stop 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 shutdown_connections 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x564987821d70 0x564987824160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564987809ba0 0x564987806ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x5649878068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/1743469652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fd2f0 0x564987809470 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 >> 172.21.15.67:0/1743469652 conn(0x5649877fb120 msgr2=0x564987800cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.435+0000 7f4d63400640 0 client.15855 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.436+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 shutdown_connections 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.436+0000 7f4d8b68c740 1 -- 172.21.15.67:0/1743469652 wait complete. 2024-09-10T23:04:03.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.436+0000 7f4d8b68c740 20 client.15855 trim_cache size 0 max 16384 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.457+0000 7f4d8b68c740 1 Processor -- start 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.457+0000 7f4d8b68c740 1 -- start start 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.457+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x564987806150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x56498780af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 0x56498780b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841b0240 con 0x564987809ba0 2024-09-10T23:04:03.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841adb10 con 0x5649877fdc10 2024-09-10T23:04:03.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ab9a0 con 0x5649877fd2f0 2024-09-10T23:04:03.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 0x56498780b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d81400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x56498780af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 0x56498780b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45210/0 (socket says 172.21.15.67:45210) 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d82800640 1 -- 172.21.15.67:0/3967444260 learned_addr learned my addr 172.21.15.67:0/3967444260 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.458+0000 7f4d82800640 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x564987806150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 -- 172.21.15.67:0/3967444260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 msgr2=0x564987806150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x564987806150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 -- 172.21.15.67:0/3967444260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 msgr2=0x56498780b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 0x56498780b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 -- 172.21.15.67:0/3967444260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564987804dc0 con 0x5649877fdc10 2024-09-10T23:04:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d81400640 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x56498780af90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d7002a930 tx=0x7f4d70005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a885180151c42d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d6e600640 1 -- 172.21.15.67:0/3967444260 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70048020 con 0x5649877fdc10 2024-09-10T23:04:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d6e600640 1 -- 172.21.15.67:0/3967444260 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d700046c0 con 0x5649877fdc10 2024-09-10T23:04:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.459+0000 7f4d6e600640 1 -- 172.21.15.67:0/3967444260 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70035040 con 0x5649877fdc10 2024-09-10T23:04:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3967444260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 msgr2=0x56498780af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x56498780af90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d7002a930 tx=0x7f4d70005e80 comp rx=0 tx=0).stop 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3967444260 shutdown_connections 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564987809ba0 0x56498780b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5649877fdc10 0x56498780af90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3967444260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x564987806150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3967444260 >> 172.21.15.67:0/3967444260 conn(0x5649877fb120 msgr2=0x56498781d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3967444260 shutdown_connections 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.460+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3967444260 wait complete. 2024-09-10T23:04:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.461+0000 7f4d8b68c740 1 -- start start 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.461+0000 7f4d8b68c740 1 Processor -- start 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.461+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x56498781ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.461+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.461+0000 7f4d8b68c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 0x564987815a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841abff0 con 0x5649877fdc10 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ad440 con 0x564987809ba0 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d8b68c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841ac6a0 con 0x5649877fd2f0 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d8b68c740 10 client.0 _ll_register_callbacks cb 0x7f4d7c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45220/0 (socket says 172.21.15.67:45220) 2024-09-10T23:04:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 -- 172.21.15.67:0/3987715080 learned_addr learned my addr 172.21.15.67:0/3987715080 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d82800640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x56498781ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d82800640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 0x564987815a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 -- 172.21.15.67:0/3987715080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 msgr2=0x56498781ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x56498781ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 -- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 msgr2=0x564987815a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 0x564987815a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d81400640 1 -- 172.21.15.67:0/3987715080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498788e930 con 0x5649877fdc10 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.462+0000 7f4d82800640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 0x564987815a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.463+0000 7f4d81400640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d700042f0 tx=0x7f4d70004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5f95973d5b13f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.463+0000 7f4d63400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.463+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70048030 con 0x5649877fdc10 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.463+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7f4d70004870 con 0x5649877fdc10 2024-09-10T23:04:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.463+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d700046c0 con 0x5649877fdc10 2024-09-10T23:04:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4d70051070 con 0x5649877fdc10 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 10 client.15861 fetch_fsmap learned FSMap version 31 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56498788fb70 con 0x5649877fdc10 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d700046c0 con 0x5649877fdc10 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4d70003c90 con 0x5649877fdc10 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 10 client.15861 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 10 client.15861 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 20 client.15861 populate_metadata read hostname 'smithi067' 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5649878902e0 0x564987892700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 4 client.15861 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.464+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5649877706a0 con 0x5649878902e0 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.465+0000 7f4d82800640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5649878902e0 0x564987892700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.465+0000 7f4d63400640 10 client.15861 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.465+0000 7f4d82800640 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5649878902e0 0x564987892700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe20c3423983019 server_cookie=edd53a5c5c83fa55 in_seq=0 out_seq=0 2024-09-10T23:04:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:03 smithi067 ceph-mon[30939]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d63400640 1 -- 172.21.15.67:0/3987715080 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649877706a0 con 0x5649878902e0 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d63400640 10 client.15861 handle_command_reply: tid=0 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 client.15861 shutdown 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 msgr2=0x564987815330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d700042f0 tx=0x7f4d70004320 comp rx=0 tx=0).stop 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 shutdown_connections 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5649878902e0 0x564987892700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564987809ba0 0x564987815a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649877fdc10 0x564987815330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 --2- 172.21.15.67:0/3987715080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649877fd2f0 0x56498781ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 >> 172.21.15.67:0/3987715080 conn(0x5649877fb120 msgr2=0x564987805960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d63400640 0 client.15861 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 shutdown_connections 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 1 -- 172.21.15.67:0/3987715080 wait complete. 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.568+0000 7f4d8b68c740 20 client.15861 trim_cache size 0 max 16384 2024-09-10T23:04:03.570 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 18, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "78941158": { 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 18, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:04:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:03.572 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 msgr2=0x7f4d680b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 0x7f4d680b7a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5649877fc000 tx=0x7f4d74000bc0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 msgr2=0x7f4d8419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d950 tx=0x7f4d7400de10 comp rx=0 tx=0).stop 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 shutdown_connections 2024-09-10T23:04:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f4d680b5580 0x7f4d680b7a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8410f200 0x7f4d841a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d84075e00 0x7f4d8419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 --2- 172.21.15.67:0/741476007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d840754e0 0x7f4d8419b190 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 >> 172.21.15.67:0/741476007 conn(0x7f4d840fe660 msgr2=0x7f4d84111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 shutdown_connections 2024-09-10T23:04:03.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:03.569+0000 7f4d83200640 1 -- 172.21.15.67:0/741476007 wait complete. 2024-09-10T23:04:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:03 smithi092 ceph-mon[34223]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-10T23:04:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:03 smithi145 ceph-mon[34397]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-10T23:04:04.491 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '78941158', quiesce took 4.5 seconds. Will release after: 25.5 2024-09-10T23:04:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:05 smithi092 ceph-mon[34223]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-09-10T23:04:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:05 smithi067 ceph-mon[30939]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-09-10T23:04:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:05 smithi145 ceph-mon[34397]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-09-10T23:04:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:07 smithi092 ceph-mon[34223]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 23 op/s 2024-09-10T23:04:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:07 smithi067 ceph-mon[30939]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 23 op/s 2024-09-10T23:04:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:07 smithi145 ceph-mon[34397]: pgmap v544: 129 pgs: 129 active+clean; 1.6 GiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 23 op/s 2024-09-10T23:04:09.682 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:09 smithi145 ceph-mon[34397]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 21 op/s 2024-09-10T23:04:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:09 smithi092 ceph-mon[34223]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 21 op/s 2024-09-10T23:04:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:09 smithi067 ceph-mon[30939]: pgmap v545: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 21 op/s 2024-09-10T23:04:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:11 smithi092 ceph-mon[34223]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-09-10T23:04:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:11 smithi067 ceph-mon[30939]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-09-10T23:04:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:11 smithi145 ceph-mon[34397]: pgmap v546: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-09-10T23:04:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:13 smithi092 ceph-mon[34223]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 23 op/s 2024-09-10T23:04:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:13 smithi067 ceph-mon[30939]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 23 op/s 2024-09-10T23:04:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:13 smithi145 ceph-mon[34397]: pgmap v547: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 23 op/s 2024-09-10T23:04:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:15 smithi092 ceph-mon[34223]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 10 op/s 2024-09-10T23:04:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:15 smithi067 ceph-mon[30939]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 10 op/s 2024-09-10T23:04:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:15 smithi145 ceph-mon[34397]: pgmap v548: 129 pgs: 129 active+clean; 1.6 GiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 MiB/s wr, 10 op/s 2024-09-10T23:04:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:17 smithi092 ceph-mon[34223]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 MiB/s wr, 8 op/s 2024-09-10T23:04:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:17 smithi067 ceph-mon[30939]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 MiB/s wr, 8 op/s 2024-09-10T23:04:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:17 smithi145 ceph-mon[34397]: pgmap v549: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 MiB/s wr, 8 op/s 2024-09-10T23:04:19.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:19 smithi067 ceph-mon[30939]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 857 KiB/s wr, 5 op/s 2024-09-10T23:04:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:19 smithi092 ceph-mon[34223]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 857 KiB/s wr, 5 op/s 2024-09-10T23:04:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:19 smithi145 ceph-mon[34397]: pgmap v550: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 857 KiB/s wr, 5 op/s 2024-09-10T23:04:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:21 smithi092 ceph-mon[34223]: pgmap v551: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 516 KiB/s wr, 7 op/s 2024-09-10T23:04:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:21 smithi067 ceph-mon[30939]: pgmap v551: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 516 KiB/s wr, 7 op/s 2024-09-10T23:04:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:21 smithi145 ceph-mon[34397]: pgmap v551: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 516 KiB/s wr, 7 op/s 2024-09-10T23:04:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:23 smithi092 ceph-mon[34223]: pgmap v552: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:23 smithi067 ceph-mon[30939]: pgmap v552: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:23 smithi145 ceph-mon[34397]: pgmap v552: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:25 smithi092 ceph-mon[34223]: pgmap v553: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:25 smithi067 ceph-mon[30939]: pgmap v553: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:25 smithi145 ceph-mon[34397]: pgmap v553: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:04:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:27 smithi092 ceph-mon[34223]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:04:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:27 smithi067 ceph-mon[30939]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:04:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:27 smithi145 ceph-mon[34397]: pgmap v554: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:04:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:29 smithi092 ceph-mon[34223]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:04:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:29 smithi067 ceph-mon[30939]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:04:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:29 smithi145 ceph-mon[34397]: pgmap v555: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:04:29.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:04:30.387 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 -- 172.21.15.67:0/2697679411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c103cc0 msgr2=0x7fbb9c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2697679411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c1040a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009900 tx=0x7fbb8c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 -- 172.21.15.67:0/2697679411 shutdown_connections 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2697679411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 0x7fbb9c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2697679411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2697679411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c1040a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 -- 172.21.15.67:0/2697679411 >> 172.21.15.67:0/2697679411 conn(0x7fbb9c0fd700 msgr2=0x7fbb9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 -- 172.21.15.67:0/2697679411 shutdown_connections 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.970+0000 7fbba2e00640 1 -- 172.21.15.67:0/2697679411 wait complete. 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.971+0000 7fbba2e00640 1 Processor -- start 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.971+0000 7fbba2e00640 1 -- start start 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 0x7fbb9c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c112c90 con 0x7fbb9c1045e0 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c112b10 con 0x7fbb9c103cc0 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c112e10 con 0x7fbb9c10d120 2024-09-10T23:04:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbb93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbb93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58446/0 (socket says 172.21.15.67:58446) 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbb93400640 1 -- 172.21.15.67:0/2562984443 learned_addr learned my addr 172.21.15.67:0/2562984443 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbb9be00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.972+0000 7fbba0e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 0x7fbb9c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 -- 172.21.15.67:0/2562984443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 msgr2=0x7fbb9c1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 0x7fbb9c1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 -- 172.21.15.67:0/2562984443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c103cc0 msgr2=0x7fbb9c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 -- 172.21.15.67:0/2562984443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1a75e0 con 0x7fbb9c1045e0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb93400640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400d740 tx=0x7fbb8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84017020 con 0x7fbb9c1045e0 2024-09-10T23:04:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb8400de50 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400f620 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c1a78d0 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.973+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1a7dc0 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.974+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c1040a0 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.978+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbb84010460 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.980+0000 7fbb99600640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 0x7fbb800b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.980+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fbb840e6930 con 0x7fbb9c1045e0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.980+0000 7fbb9be00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 0x7fbb800b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.981+0000 7fbb9be00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 0x7fbb800b7990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006010 tx=0x7fbb8c005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:31.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:30.982+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbb84021020 con 0x7fbb9c1045e0 2024-09-10T23:04:31.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.218+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb9c108140 con 0x7fbb9c1045e0 2024-09-10T23:04:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.219+0000 7fbb99600640 1 -- 172.21.15.67:0/2562984443 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fbb840ad140 con 0x7fbb9c1045e0 2024-09-10T23:04:31.221 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:04:31.221 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:04:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.222+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 msgr2=0x7fbb800b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.222+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 0x7fbb800b7990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006010 tx=0x7fbb8c005e80 comp rx=0 tx=0).stop 2024-09-10T23:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.222+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 msgr2=0x7fbb9c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.222+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400d740 tx=0x7fbb8400dc00 comp rx=0 tx=0).stop 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 shutdown_connections 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbb800b54d0 0x7fbb800b7990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb9c10d120 0x7fbb9c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb9c1045e0 0x7fbb9c101eb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 --2- 172.21.15.67:0/2562984443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb9c103cc0 0x7fbb9c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 >> 172.21.15.67:0/2562984443 conn(0x7fbb9c0fd700 msgr2=0x7fbb9c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 shutdown_connections 2024-09-10T23:04:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:31.223+0000 7fbba2e00640 1 -- 172.21.15.67:0/2562984443 wait complete. 2024-09-10T23:04:31.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:31 smithi067 ceph-mon[30939]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:31.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:31 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2562984443' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:31 smithi092 ceph-mon[34223]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:31 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2562984443' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:31 smithi145 ceph-mon[34397]: pgmap v556: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:04:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:31 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2562984443' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:32.246 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 78941158 --release --await' 2024-09-10T23:04:32.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 78941158 --release --await 2024-09-10T23:04:32.629 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 -- 172.21.15.67:0/3879482217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb13010a9b0 msgr2=0x7fb13010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 --2- 172.21.15.67:0/3879482217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb13010a9b0 0x7fb13010adf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb118009900 tx=0x7fb11802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 -- 172.21.15.67:0/3879482217 shutdown_connections 2024-09-10T23:04:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 --2- 172.21.15.67:0/3879482217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 0x7fb130111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 --2- 172.21.15.67:0/3879482217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb13010a9b0 0x7fb13010adf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 --2- 172.21.15.67:0/3879482217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb130104fa0 0x7fb130105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 -- 172.21.15.67:0/3879482217 >> 172.21.15.67:0/3879482217 conn(0x7fb130100a50 msgr2=0x7fb130102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.262+0000 7fb134a00640 1 -- 172.21.15.67:0/3879482217 shutdown_connections 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.263+0000 7fb134a00640 1 -- 172.21.15.67:0/3879482217 wait complete. 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 Processor -- start 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 -- start start 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 0x7fb13019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 0x7fb1301a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb130114de0 con 0x7fb130104fa0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb130114c60 con 0x7fb13010a9b0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.264+0000 7fb134a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb130114f60 con 0x7fb13010b330 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58462/0 (socket says 172.21.15.67:58462) 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 -- 172.21.15.67:0/2608179599 learned_addr learned my addr 172.21.15.67:0/2608179599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12e800640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 0x7fb1301a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 -- 172.21.15.67:0/2608179599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 msgr2=0x7fb1301a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 0x7fb1301a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 -- 172.21.15.67:0/2608179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 msgr2=0x7fb13019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb125400640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 0x7fb13019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 0x7fb13019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 -- 172.21.15.67:0/2608179599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1301a1420 con 0x7fb130104fa0 2024-09-10T23:04:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb125400640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 0x7fb13019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.265+0000 7fb12de00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ebc0 tx=0x7fb12000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.266+0000 7fb127400640 1 -- 172.21.15.67:0/2608179599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120010630 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.266+0000 7fb127400640 1 -- 172.21.15.67:0/2608179599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb120010c60 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.266+0000 7fb127400640 1 -- 172.21.15.67:0/2608179599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120018b00 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.266+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1301a1710 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.266+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a7dd0 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 Processor -- start 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 -- start start 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb127400640 1 -- 172.21.15.67:0/2608179599 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb120018ca0 con 0x7fb130104fa0 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c794483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 0x561c79456c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561c793bf290 con 0x561c79448910 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.269+0000 7fb1373e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c793ae6c0 con 0x561c794548a0 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb1373e9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c793b24d0 con 0x561c79447ff0 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58464/0 (socket says 172.21.15.67:58464) 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 -- 172.21.15.67:0/940064438 learned_addr learned my addr 172.21.15.67:0/940064438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12e800640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c794483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb125400640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 0x561c79456c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 -- 172.21.15.67:0/940064438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 msgr2=0x561c794483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c794483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 -- 172.21.15.67:0/940064438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 msgr2=0x561c79456c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 0x561c79456c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb12de00640 1 -- 172.21.15.67:0/940064438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7933b460 con 0x561c79448910 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.270+0000 7fb125400640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 0x561c79456c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb12de00640 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb1200eab60 tx=0x7fb1200ef540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ea6da179debf31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb11f000640 1 -- 172.21.15.67:0/940064438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200f0040 con 0x561c79448910 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb11f000640 1 -- 172.21.15.67:0/940064438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1200efc50 con 0x561c79448910 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb11f000640 1 -- 172.21.15.67:0/940064438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200f3390 con 0x561c79448910 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb127400640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 0x7fb1140b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.271+0000 7fb127400640 1 -- 172.21.15.67:0/2608179599 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fb1200e6a80 con 0x7fb130104fa0 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb12e800640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 0x7fb1140b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 -- 172.21.15.67:0/940064438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 msgr2=0x561c79454360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb1200eab60 tx=0x7fb1200ef540 comp rx=0 tx=0).stop 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 -- 172.21.15.67:0/940064438 shutdown_connections 2024-09-10T23:04:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c794548a0 0x561c79456c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79448910 0x561c79454360 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 --2- 172.21.15.67:0/940064438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c794483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 -- 172.21.15.67:0/940064438 >> 172.21.15.67:0/940064438 conn(0x561c79445e20 msgr2=0x561c79446fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 -- 172.21.15.67:0/940064438 shutdown_connections 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb12e800640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 0x7fb1140b7990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cc60 tx=0x7fb12800c430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.272+0000 7fb1373e9740 1 -- 172.21.15.67:0/940064438 wait complete. 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 -- start start 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 Processor -- start 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 0x561c79451dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561c7920fbb0 con 0x561c794548a0 2024-09-10T23:04:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c793496c0 con 0x561c79447ff0 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.273+0000 7fb1373e9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c79457550 con 0x561c79448910 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb1373e9740 10 client.0 _ll_register_callbacks cb 0x561c79364490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:33032/0 (socket says 172.21.15.67:33032) 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12de00640 1 -- 172.21.15.67:0/352063007 learned_addr learned my addr 172.21.15.67:0/352063007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58468/0 (socket says 172.21.15.67:58468) 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb125400640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 0x561c79451dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 -- 172.21.15.67:0/352063007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 msgr2=0x561c79451dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 0x561c79451dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 -- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 msgr2=0x561c79451880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12e800640 1 -- 172.21.15.67:0/352063007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7910cb00 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb12de00640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.274+0000 7fb125400640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 0x561c79451dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb12e800640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eda0 tx=0x7fb1280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ee660a3d22a3da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb113400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12801cb20 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fb12801ccc0 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1200efc50 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb128031050 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb1373e9740 10 client.15879 fetch_fsmap learned FSMap version 31 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.275+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c79183960 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1200efc50 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb12801a390 con 0x561c794548a0 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 10 client.15879 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:04:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 10 client.15879 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:04:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 20 client.15879 populate_metadata read hostname 'smithi067' 2024-09-10T23:04:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794644d0 0x561c794668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 4 client.15879 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.276+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c793c11e0 con 0x561c794644d0 2024-09-10T23:04:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.277+0000 7fb12de00640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794644d0 0x561c794668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.277+0000 7fb12de00640 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794644d0 0x561c794668f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1905b54927a5d6f server_cookie=cbb88dfa5403dd1 in_seq=0 out_seq=0 2024-09-10T23:04:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.277+0000 7fb113400640 10 client.15879 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb113400640 1 -- 172.21.15.67:0/352063007 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c793c11e0 con 0x561c794644d0 2024-09-10T23:04:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb113400640 10 client.15879 handle_command_reply: tid=0 2024-09-10T23:04:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 client.15879 shutdown 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 msgr2=0x561c79452300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eda0 tx=0x7fb1280046c0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 shutdown_connections 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794644d0 0x561c794668f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c79452300 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79448910 0x561c79451dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 --2- 172.21.15.67:0/352063007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79447ff0 0x561c79451880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 >> 172.21.15.67:0/352063007 conn(0x561c79445e20 msgr2=0x561c7944b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.278+0000 7fb113400640 0 client.15879 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.279+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 shutdown_connections 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.279+0000 7fb1373e9740 1 -- 172.21.15.67:0/352063007 wait complete. 2024-09-10T23:04:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.279+0000 7fb1373e9740 20 client.15879 trim_cache size 0 max 16384 2024-09-10T23:04:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 Processor -- start 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 -- start start 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c7944ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c7944efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c7946c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b48e0 con 0x561c794548a0 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b21b0 con 0x561c79448910 2024-09-10T23:04:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb1373e9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b0000 con 0x561c79447ff0 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.297+0000 7fb125400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c7944efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb125400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c7944efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:33042/0 (socket says 172.21.15.67:33042) 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c7944ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb125400640 1 -- 172.21.15.67:0/78897139 learned_addr learned my addr 172.21.15.67:0/78897139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12e800640 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c7946c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 -- 172.21.15.67:0/78897139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 msgr2=0x561c7944efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c7944efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 -- 172.21.15.67:0/78897139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 msgr2=0x561c7946c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c7946c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 -- 172.21.15.67:0/78897139 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c79448740 con 0x561c79447ff0 2024-09-10T23:04:33.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.298+0000 7fb12de00640 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c7944ea90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb12000eb90 tx=0x7fb12000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5411aa97d76874c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.299+0000 7fb11e600640 1 -- 172.21.15.67:0/78897139 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200a9160 con 0x561c79447ff0 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.299+0000 7fb11e600640 1 -- 172.21.15.67:0/78897139 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb120018420 con 0x561c79447ff0 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.299+0000 7fb11e600640 1 -- 172.21.15.67:0/78897139 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200a8280 con 0x561c79447ff0 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- 172.21.15.67:0/78897139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 msgr2=0x561c7944ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c7944ea90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb12000eb90 tx=0x7fb12000cf90 comp rx=0 tx=0).stop 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- 172.21.15.67:0/78897139 shutdown_connections 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c794548a0 0x561c7946c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c7944efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 --2- 172.21.15.67:0/78897139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c79447ff0 0x561c7944ea90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- 172.21.15.67:0/78897139 >> 172.21.15.67:0/78897139 conn(0x561c79445e20 msgr2=0x561c79446d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- 172.21.15.67:0/78897139 shutdown_connections 2024-09-10T23:04:33.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- 172.21.15.67:0/78897139 wait complete. 2024-09-10T23:04:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.300+0000 7fb1373e9740 1 -- start start 2024-09-10T23:04:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 Processor -- start 2024-09-10T23:04:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c79459880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c794548a0 0x561c794ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b0670 con 0x561c79447ff0 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b1ae0 con 0x561c79448910 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.301+0000 7fb1373e9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb1301b0d40 con 0x561c794548a0 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb1373e9740 10 client.0 _ll_register_callbacks cb 0x7fb128005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58490/0 (socket says 172.21.15.67:58490) 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 -- 172.21.15.67:0/1241907198 learned_addr learned my addr 172.21.15.67:0/1241907198 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:04:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb125400640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c79459880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12e800640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c794548a0 0x561c794ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 -- 172.21.15.67:0/1241907198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c794548a0 msgr2=0x561c794ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c794548a0 0x561c794ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 -- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 msgr2=0x561c79459880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c79459880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.302+0000 7fb12de00640 1 -- 172.21.15.67:0/1241907198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c794cac60 con 0x561c79447ff0 2024-09-10T23:04:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb12de00640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb1200e6040 tx=0x7fb12000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6318ca24549c884c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb113400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200d3020 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9431+0+0 (secure 0 0 0) 0x7fb1200b1170 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c793bcc90 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb1200cfe60 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb1373e9740 10 client.15885 fetch_fsmap learned FSMap version 31 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.303+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c794d2120 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561c793bcc90 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb1200f3070 con 0x561c79447ff0 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 10 client.15885 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 10 client.15885 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 20 client.15885 populate_metadata read hostname 'smithi067' 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794d23a0 0x561c794d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 4 client.15885 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "78941158", "release": true, "await": true}] 2024-09-10T23:04:33.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.304+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "78941158", "release": true, "await": true}) -- 0x561c793ba0c0 con 0x561c794d23a0 2024-09-10T23:04:33.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.305+0000 7fb125400640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794d23a0 0x561c794d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:33.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.305+0000 7fb125400640 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794d23a0 0x561c794d4790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917b8b2fc5f22ca6 server_cookie=ecac71d0281a1b07 in_seq=0 out_seq=0 2024-09-10T23:04:33.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.305+0000 7fb113400640 10 client.15885 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.329+0000 7fb113400640 1 -- 172.21.15.67:0/1241907198 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c793ba0c0 con 0x561c794d23a0 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.329+0000 7fb113400640 10 client.15885 handle_command_reply: tid=0 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.329+0000 7fb1373e9740 1 client.15885 shutdown 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 msgr2=0x561c794592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb1200e6040 tx=0x7fb12000cd60 comp rx=0 tx=0).stop 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 shutdown_connections 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561c794d23a0 0x561c794d4790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c794548a0 0x561c794ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c79448910 0x561c79459880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 --2- 172.21.15.67:0/1241907198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c79447ff0 0x561c794592e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 >> 172.21.15.67:0/1241907198 conn(0x561c79445e20 msgr2=0x561c79452eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb113400640 0 client.15885 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 shutdown_connections 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 1 -- 172.21.15.67:0/1241907198 wait complete. 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.330+0000 7fb1373e9740 20 client.15885 trim_cache size 0 max 16384 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:04:33.333 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 20, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "78941158": { 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 20, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:04:33.334 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:04:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 msgr2=0x7fb1140b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 0x7fb1140b7990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cc60 tx=0x7fb12800c430 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 msgr2=0x7fb13019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ebc0 tx=0x7fb12000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 shutdown_connections 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb1140b54d0 0x7fb1140b7990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb13010b330 0x7fb1301a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb13010a9b0 0x7fb13019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 --2- 172.21.15.67:0/2608179599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb130104fa0 0x7fb13019b370 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.331+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 >> 172.21.15.67:0/2608179599 conn(0x7fb130100a50 msgr2=0x7fb130102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.332+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 shutdown_connections 2024-09-10T23:04:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:04:33.332+0000 7fb134a00640 1 -- 172.21.15.67:0/2608179599 wait complete. 2024-09-10T23:04:33.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:33 smithi092 ceph-mon[34223]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:04:33.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:33 smithi067 ceph-mon[30939]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:04:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:33 smithi145 ceph-mon[34397]: pgmap v557: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:04:34.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '78941158', total seconds elapsed: 34.5 2024-09-10T23:04:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:35 smithi092 ceph-mon[34223]: pgmap v558: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:04:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:35 smithi067 ceph-mon[30939]: pgmap v558: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:04:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:35 smithi145 ceph-mon[34397]: pgmap v558: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:04:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:37 smithi092 ceph-mon[34223]: pgmap v559: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 510 KiB/s wr, 6 op/s 2024-09-10T23:04:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:37 smithi067 ceph-mon[30939]: pgmap v559: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 510 KiB/s wr, 6 op/s 2024-09-10T23:04:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:37 smithi145 ceph-mon[34397]: pgmap v559: 129 pgs: 129 active+clean; 1.6 GiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 510 KiB/s wr, 6 op/s 2024-09-10T23:04:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:39 smithi092 ceph-mon[34223]: pgmap v560: 129 pgs: 129 active+clean; 1.6 GiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:04:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:39 smithi067 ceph-mon[30939]: pgmap v560: 129 pgs: 129 active+clean; 1.6 GiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:04:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:39 smithi145 ceph-mon[34397]: pgmap v560: 129 pgs: 129 active+clean; 1.6 GiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-09-10T23:04:40.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:40 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:40.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:40 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:40.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:40 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: pgmap v561: 129 pgs: 129 active+clean; 1.6 GiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 9 op/s 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: pgmap v561: 129 pgs: 129 active+clean; 1.6 GiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 9 op/s 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:41.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:04:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: pgmap v561: 129 pgs: 129 active+clean; 1.6 GiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 9 op/s 2024-09-10T23:04:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:04:41.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:41.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:41.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:04:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:42 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:04:42.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:42 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:04:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:42 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:04:42.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:42 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:04:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:42 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:04:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:42 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:04:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:43 smithi092 ceph-mon[34223]: pgmap v562: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-10T23:04:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:43 smithi067 ceph-mon[30939]: pgmap v562: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-10T23:04:43.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:43 smithi145 ceph-mon[34397]: pgmap v562: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 MiB/s wr, 8 op/s 2024-09-10T23:04:45.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:45 smithi092 ceph-mon[34223]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:04:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:45 smithi067 ceph-mon[30939]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:04:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:45 smithi145 ceph-mon[34397]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:04:47.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:47 smithi092 ceph-mon[34223]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:04:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:47 smithi067 ceph-mon[30939]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:04:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:47 smithi145 ceph-mon[34397]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:04:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:49 smithi092 ceph-mon[34223]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:04:49.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:49 smithi067 ceph-mon[30939]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:04:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:49 smithi145 ceph-mon[34397]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:04:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:51 smithi092 ceph-mon[34223]: pgmap v566: 129 pgs: 129 active+clean; 1.8 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:04:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:51 smithi067 ceph-mon[30939]: pgmap v566: 129 pgs: 129 active+clean; 1.8 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:04:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:51 smithi145 ceph-mon[34397]: pgmap v566: 129 pgs: 129 active+clean; 1.8 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:04:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:53 smithi092 ceph-mon[34223]: pgmap v567: 129 pgs: 129 active+clean; 1.8 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:04:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:53 smithi067 ceph-mon[30939]: pgmap v567: 129 pgs: 129 active+clean; 1.8 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:04:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:53 smithi145 ceph-mon[34397]: pgmap v567: 129 pgs: 129 active+clean; 1.8 GiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:04:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:55 smithi092 ceph-mon[34223]: pgmap v568: 129 pgs: 129 active+clean; 1.8 GiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:04:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:55 smithi067 ceph-mon[30939]: pgmap v568: 129 pgs: 129 active+clean; 1.8 GiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:04:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:55 smithi145 ceph-mon[34397]: pgmap v568: 129 pgs: 129 active+clean; 1.8 GiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:04:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:57 smithi092 ceph-mon[34223]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:04:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:57 smithi067 ceph-mon[30939]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:04:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:57 smithi145 ceph-mon[34397]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:04:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:04:59 smithi092 ceph-mon[34223]: pgmap v570: 129 pgs: 129 active+clean; 1.9 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:04:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:04:59 smithi067 ceph-mon[30939]: pgmap v570: 129 pgs: 129 active+clean; 1.9 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:04:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:04:59 smithi145 ceph-mon[34397]: pgmap v570: 129 pgs: 129 active+clean; 1.9 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:05:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:01 smithi092 ceph-mon[34223]: pgmap v571: 129 pgs: 129 active+clean; 1.9 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:05:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:01 smithi067 ceph-mon[30939]: pgmap v571: 129 pgs: 129 active+clean; 1.9 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:05:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:01 smithi145 ceph-mon[34397]: pgmap v571: 129 pgs: 129 active+clean; 1.9 GiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:05:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:03 smithi092 ceph-mon[34223]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-09-10T23:05:03.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:03 smithi067 ceph-mon[30939]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-09-10T23:05:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:03 smithi145 ceph-mon[34397]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-09-10T23:05:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:05 smithi092 ceph-mon[34223]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 28 op/s 2024-09-10T23:05:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:05 smithi067 ceph-mon[30939]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 28 op/s 2024-09-10T23:05:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:05 smithi145 ceph-mon[34397]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 28 op/s 2024-09-10T23:05:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:07 smithi092 ceph-mon[34223]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:05:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:07 smithi067 ceph-mon[30939]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:05:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:07 smithi145 ceph-mon[34397]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 28 op/s 2024-09-10T23:05:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:09 smithi067 ceph-mon[30939]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:05:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:09 smithi145 ceph-mon[34397]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:05:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:09 smithi092 ceph-mon[34223]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:05:11.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:11 smithi067 ceph-mon[30939]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-10T23:05:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:11 smithi145 ceph-mon[34397]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-10T23:05:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:11 smithi092 ceph-mon[34223]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-10T23:05:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:13 smithi067 ceph-mon[30939]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:05:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:13 smithi145 ceph-mon[34397]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:05:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:13 smithi092 ceph-mon[34223]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:05:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:15 smithi067 ceph-mon[30939]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:05:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:15 smithi145 ceph-mon[34397]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:05:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:15 smithi092 ceph-mon[34223]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:05:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:17 smithi067 ceph-mon[30939]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:17.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:17 smithi092 ceph-mon[34223]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:17 smithi145 ceph-mon[34397]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:19 smithi067 ceph-mon[30939]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:19 smithi145 ceph-mon[34397]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:19 smithi092 ceph-mon[34223]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:21 smithi067 ceph-mon[30939]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:21 smithi145 ceph-mon[34397]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:21 smithi092 ceph-mon[34223]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:23 smithi067 ceph-mon[30939]: pgmap v582: 129 pgs: 129 active+clean; 2.3 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:23 smithi145 ceph-mon[34397]: pgmap v582: 129 pgs: 129 active+clean; 2.3 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:23 smithi092 ceph-mon[34223]: pgmap v582: 129 pgs: 129 active+clean; 2.3 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:25 smithi067 ceph-mon[30939]: pgmap v583: 129 pgs: 129 active+clean; 2.3 GiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:25 smithi145 ceph-mon[34397]: pgmap v583: 129 pgs: 129 active+clean; 2.3 GiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:25 smithi092 ceph-mon[34223]: pgmap v583: 129 pgs: 129 active+clean; 2.3 GiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:27 smithi145 ceph-mon[34397]: pgmap v584: 129 pgs: 129 active+clean; 2.3 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:27 smithi092 ceph-mon[34223]: pgmap v584: 129 pgs: 129 active+clean; 2.3 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:27 smithi067 ceph-mon[30939]: pgmap v584: 129 pgs: 129 active+clean; 2.3 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:29 smithi145 ceph-mon[34397]: pgmap v585: 129 pgs: 129 active+clean; 2.3 GiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:29 smithi092 ceph-mon[34223]: pgmap v585: 129 pgs: 129 active+clean; 2.3 GiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:29 smithi067 ceph-mon[30939]: pgmap v585: 129 pgs: 129 active+clean; 2.3 GiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:31 smithi145 ceph-mon[34397]: pgmap v586: 129 pgs: 129 active+clean; 2.4 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:31 smithi092 ceph-mon[34223]: pgmap v586: 129 pgs: 129 active+clean; 2.4 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:31 smithi067 ceph-mon[30939]: pgmap v586: 129 pgs: 129 active+clean; 2.4 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:33 smithi145 ceph-mon[34397]: pgmap v587: 129 pgs: 129 active+clean; 2.4 GiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:33 smithi092 ceph-mon[34223]: pgmap v587: 129 pgs: 129 active+clean; 2.4 GiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:33 smithi067 ceph-mon[30939]: pgmap v587: 129 pgs: 129 active+clean; 2.4 GiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:35 smithi145 ceph-mon[34397]: pgmap v588: 129 pgs: 129 active+clean; 2.4 GiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:05:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:35 smithi092 ceph-mon[34223]: pgmap v588: 129 pgs: 129 active+clean; 2.4 GiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:05:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:35 smithi067 ceph-mon[30939]: pgmap v588: 129 pgs: 129 active+clean; 2.4 GiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:05:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:37 smithi145 ceph-mon[34397]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:37 smithi092 ceph-mon[34223]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:37 smithi067 ceph-mon[30939]: pgmap v589: 129 pgs: 129 active+clean; 2.5 GiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:05:39.850 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:39 smithi067 ceph-mon[30939]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:05:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:39 smithi145 ceph-mon[34397]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:05:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:39 smithi092 ceph-mon[34223]: pgmap v590: 129 pgs: 129 active+clean; 2.5 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:05:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:41 smithi145 ceph-mon[34397]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:41 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:41 smithi092 ceph-mon[34223]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:41 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:41 smithi067 ceph-mon[30939]: pgmap v591: 129 pgs: 129 active+clean; 2.5 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:41 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:42.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:43.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:43.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:05:43.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:43.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:43.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:05:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:43 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:05:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:43 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:05:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:43 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:05:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:43 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:05:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:43 smithi145 ceph-mon[34397]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:05:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:43 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:05:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:43 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:05:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:43 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:05:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:43 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:05:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:43 smithi092 ceph-mon[34223]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:05:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:43 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:05:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:43 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:05:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:43 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:05:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:43 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:05:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:43 smithi067 ceph-mon[30939]: pgmap v592: 129 pgs: 129 active+clean; 2.5 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:05:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:45 smithi145 ceph-mon[34397]: pgmap v593: 129 pgs: 129 active+clean; 2.6 GiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:05:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:45 smithi092 ceph-mon[34223]: pgmap v593: 129 pgs: 129 active+clean; 2.6 GiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:05:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:45 smithi067 ceph-mon[30939]: pgmap v593: 129 pgs: 129 active+clean; 2.6 GiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:05:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:47 smithi145 ceph-mon[34397]: pgmap v594: 129 pgs: 129 active+clean; 2.6 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:05:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:47 smithi092 ceph-mon[34223]: pgmap v594: 129 pgs: 129 active+clean; 2.6 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:05:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:47 smithi067 ceph-mon[30939]: pgmap v594: 129 pgs: 129 active+clean; 2.6 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:05:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:49 smithi145 ceph-mon[34397]: pgmap v595: 129 pgs: 129 active+clean; 2.6 GiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:49 smithi092 ceph-mon[34223]: pgmap v595: 129 pgs: 129 active+clean; 2.6 GiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:49 smithi067 ceph-mon[30939]: pgmap v595: 129 pgs: 129 active+clean; 2.6 GiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 10 op/s 2024-09-10T23:05:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:51 smithi145 ceph-mon[34397]: pgmap v596: 129 pgs: 129 active+clean; 2.7 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:51 smithi092 ceph-mon[34223]: pgmap v596: 129 pgs: 129 active+clean; 2.7 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:51 smithi067 ceph-mon[30939]: pgmap v596: 129 pgs: 129 active+clean; 2.7 GiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:05:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:53 smithi145 ceph-mon[34397]: pgmap v597: 129 pgs: 129 active+clean; 2.7 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:53 smithi092 ceph-mon[34223]: pgmap v597: 129 pgs: 129 active+clean; 2.7 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:53 smithi067 ceph-mon[30939]: pgmap v597: 129 pgs: 129 active+clean; 2.7 GiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:05:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:55 smithi145 ceph-mon[34397]: pgmap v598: 129 pgs: 129 active+clean; 2.7 GiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:55 smithi092 ceph-mon[34223]: pgmap v598: 129 pgs: 129 active+clean; 2.7 GiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:56.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:55 smithi067 ceph-mon[30939]: pgmap v598: 129 pgs: 129 active+clean; 2.7 GiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:05:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:57 smithi145 ceph-mon[34397]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:57 smithi092 ceph-mon[34223]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:57 smithi067 ceph-mon[30939]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:05:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:05:59 smithi145 ceph-mon[34397]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:06:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:05:59 smithi092 ceph-mon[34223]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:06:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:05:59 smithi067 ceph-mon[30939]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:06:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:01 smithi145 ceph-mon[34397]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:01 smithi092 ceph-mon[34223]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:01 smithi067 ceph-mon[30939]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:03 smithi145 ceph-mon[34397]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:06:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:03 smithi145 ceph-mon[34397]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T23:06:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:03 smithi092 ceph-mon[34223]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:06:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:03 smithi092 ceph-mon[34223]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T23:06:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:03 smithi067 ceph-mon[30939]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:06:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:03 smithi067 ceph-mon[30939]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T23:06:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:05 smithi145 ceph-mon[34397]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:06:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:05 smithi092 ceph-mon[34223]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:06:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:05 smithi067 ceph-mon[30939]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:06:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:07 smithi145 ceph-mon[34397]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:07 smithi092 ceph-mon[34223]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:07 smithi067 ceph-mon[30939]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:09 smithi145 ceph-mon[34397]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:09 smithi092 ceph-mon[34223]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:09 smithi067 ceph-mon[30939]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:11 smithi092 ceph-mon[34223]: pgmap v607: 129 pgs: 129 active+clean; 3.0 GiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:11 smithi067 ceph-mon[30939]: pgmap v607: 129 pgs: 129 active+clean; 3.0 GiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:11 smithi145 ceph-mon[34397]: pgmap v607: 129 pgs: 129 active+clean; 3.0 GiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:06:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:13 smithi092 ceph-mon[34223]: pgmap v608: 129 pgs: 129 active+clean; 3.0 GiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:06:14.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:13 smithi092 ceph-mon[34223]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T23:06:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:13 smithi067 ceph-mon[30939]: pgmap v608: 129 pgs: 129 active+clean; 3.0 GiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:06:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:13 smithi067 ceph-mon[30939]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T23:06:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:13 smithi145 ceph-mon[34397]: pgmap v608: 129 pgs: 129 active+clean; 3.0 GiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:06:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:13 smithi145 ceph-mon[34397]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T23:06:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:15 smithi092 ceph-mon[34223]: pgmap v610: 129 pgs: 129 active+clean; 3.0 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:15 smithi067 ceph-mon[30939]: pgmap v610: 129 pgs: 129 active+clean; 3.0 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:15 smithi145 ceph-mon[34397]: pgmap v610: 129 pgs: 129 active+clean; 3.0 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:17 smithi092 ceph-mon[34223]: pgmap v611: 129 pgs: 129 active+clean; 3.1 GiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:17 smithi067 ceph-mon[30939]: pgmap v611: 129 pgs: 129 active+clean; 3.1 GiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:17 smithi145 ceph-mon[34397]: pgmap v611: 129 pgs: 129 active+clean; 3.1 GiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:19 smithi092 ceph-mon[34223]: pgmap v612: 129 pgs: 129 active+clean; 3.1 GiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:06:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:19 smithi067 ceph-mon[30939]: pgmap v612: 129 pgs: 129 active+clean; 3.1 GiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:06:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:19 smithi145 ceph-mon[34397]: pgmap v612: 129 pgs: 129 active+clean; 3.1 GiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:06:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:21 smithi092 ceph-mon[34223]: pgmap v613: 129 pgs: 129 active+clean; 3.1 GiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:21 smithi067 ceph-mon[30939]: pgmap v613: 129 pgs: 129 active+clean; 3.1 GiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:21 smithi145 ceph-mon[34397]: pgmap v613: 129 pgs: 129 active+clean; 3.1 GiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:23 smithi092 ceph-mon[34223]: pgmap v614: 129 pgs: 129 active+clean; 3.1 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:23 smithi067 ceph-mon[30939]: pgmap v614: 129 pgs: 129 active+clean; 3.1 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:23 smithi145 ceph-mon[34397]: pgmap v614: 129 pgs: 129 active+clean; 3.1 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:06:25.963 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-09-10T23:06:25.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:06:26.016 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:25 smithi067 ceph-mon[30939]: pgmap v615: 129 pgs: 129 active+clean; 3.2 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:25 smithi092 ceph-mon[34223]: pgmap v615: 129 pgs: 129 active+clean; 3.2 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:25 smithi145 ceph-mon[34397]: pgmap v615: 129 pgs: 129 active+clean; 3.2 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 -- 172.21.15.67:0/2540796569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78108fa0 msgr2=0x7fba7810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 --2- 172.21.15.67:0/2540796569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78108fa0 0x7fba7810f7a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fba70009ca0 tx=0x7fba7002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 -- 172.21.15.67:0/2540796569 shutdown_connections 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 --2- 172.21.15.67:0/2540796569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78108fa0 0x7fba7810f7a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 --2- 172.21.15.67:0/2540796569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 0x7fba78108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.178+0000 7fba77e00640 1 --2- 172.21.15.67:0/2540796569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78102c10 0x7fba78102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.179+0000 7fba77e00640 1 -- 172.21.15.67:0/2540796569 >> 172.21.15.67:0/2540796569 conn(0x7fba780fe660 msgr2=0x7fba78100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.179+0000 7fba77e00640 1 -- 172.21.15.67:0/2540796569 shutdown_connections 2024-09-10T23:06:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.179+0000 7fba77e00640 1 -- 172.21.15.67:0/2540796569 wait complete. 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 Processor -- start 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 -- start start 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 0x7fba781993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78108fa0 0x7fba7819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112b60 con 0x7fba78102c10 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba781129e0 con 0x7fba78108620 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.180+0000 7fba77e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112ce0 con 0x7fba78108fa0 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40186/0 (socket says 172.21.15.67:40186) 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 -- 172.21.15.67:0/350205911 learned_addr learned my addr 172.21.15.67:0/350205911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76000640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 0x7fba781993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba77400640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78108fa0 0x7fba7819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 -- 172.21.15.67:0/350205911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78108fa0 msgr2=0x7fba7819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78108fa0 0x7fba7819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 -- 172.21.15.67:0/350205911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 msgr2=0x7fba781993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 0x7fba781993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.181+0000 7fba76a00640 1 -- 172.21.15.67:0/350205911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7819ef60 con 0x7fba78102c10 2024-09-10T23:06:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba76a00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d950 tx=0x7fba6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68017070 con 0x7fba78102c10 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba68009c30 con 0x7fba78102c10 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800f620 con 0x7fba78102c10 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7819f250 con 0x7fba78102c10 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.182+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba78075550 con 0x7fba78102c10 2024-09-10T23:06:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.183+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba78103ec0 con 0x7fba78102c10 2024-09-10T23:06:27.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.186+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fba68010460 con 0x7fba78102c10 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.187+0000 7fba6f400640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 0x7fba5c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.187+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7fba680e5fe0 con 0x7fba78102c10 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.187+0000 7fba76000640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 0x7fba5c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.188+0000 7fba76000640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 0x7fba5c0b7a40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba78108180 tx=0x7fba6400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.193+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fba680ac720 con 0x7fba78102c10 2024-09-10T23:06:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.450+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba780757b0 con 0x7fba78102c10 2024-09-10T23:06:27.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.451+0000 7fba6f400640 1 -- 172.21.15.67:0/350205911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fba68009de0 con 0x7fba78102c10 2024-09-10T23:06:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:06:27.453 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:06:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 msgr2=0x7fba5c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 0x7fba5c0b7a40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba78108180 tx=0x7fba6400b390 comp rx=0 tx=0).stop 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 msgr2=0x7fba78198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d950 tx=0x7fba6800de10 comp rx=0 tx=0).stop 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 shutdown_connections 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fba5c0b5580 0x7fba5c0b7a40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba78108fa0 0x7fba7819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba78108620 0x7fba781993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 --2- 172.21.15.67:0/350205911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba78102c10 0x7fba78198eb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 >> 172.21.15.67:0/350205911 conn(0x7fba780fe660 msgr2=0x7fba78101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.454+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 shutdown_connections 2024-09-10T23:06:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:27.455+0000 7fba77e00640 1 -- 172.21.15.67:0/350205911 wait complete. 2024-09-10T23:06:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:27 smithi067 ceph-mon[30939]: pgmap v616: 129 pgs: 129 active+clean; 3.2 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:27 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/350205911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:27 smithi092 ceph-mon[34223]: pgmap v616: 129 pgs: 129 active+clean; 3.2 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:27 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/350205911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:27 smithi145 ceph-mon[34397]: pgmap v616: 129 pgs: 129 active+clean; 3.2 GiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:06:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:27 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/350205911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:28.481 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-09-10T23:06:28.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-09-10T23:06:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.607+0000 7fc6aec00640 1 -- 172.21.15.67:0/154879893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 msgr2=0x7fc6a80758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.607+0000 7fc6aec00640 1 --2- 172.21.15.67:0/154879893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a80758a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009900 tx=0x7fc69c02a410 comp rx=0 tx=0).stop 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 -- 172.21.15.67:0/154879893 shutdown_connections 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 --2- 172.21.15.67:0/154879893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 0x7fc6a8111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 --2- 172.21.15.67:0/154879893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 0x7fc6a810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 --2- 172.21.15.67:0/154879893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a80758a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 -- 172.21.15.67:0/154879893 >> 172.21.15.67:0/154879893 conn(0x7fc6a80fe6d0 msgr2=0x7fc6a8100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 -- 172.21.15.67:0/154879893 shutdown_connections 2024-09-10T23:06:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.608+0000 7fc6aec00640 1 -- 172.21.15.67:0/154879893 wait complete. 2024-09-10T23:06:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.609+0000 7fc6aec00640 1 Processor -- start 2024-09-10T23:06:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.609+0000 7fc6aec00640 1 -- start start 2024-09-10T23:06:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 0x7fc6a8102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 0x7fc6a8103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114cf0 con 0x7fc6a80754c0 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114b70 con 0x7fc6a8075de0 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6aec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114e70 con 0x7fc6a810f210 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55014/0 (socket says 172.21.15.67:55014) 2024-09-10T23:06:29.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6ad800640 1 -- 172.21.15.67:0/3594395134 learned_addr learned my addr 172.21.15.67:0/3594395134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:29.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.610+0000 7fc6ace00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 0x7fc6a8102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ae200640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 0x7fc6a8103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 -- 172.21.15.67:0/3594395134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 msgr2=0x7fc6a8103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 0x7fc6a8103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 -- 172.21.15.67:0/3594395134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 msgr2=0x7fc6a8102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 0x7fc6a8102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 -- 172.21.15.67:0/3594395134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81ab770 con 0x7fc6a80754c0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.611+0000 7fc6ad800640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02a930 tx=0x7fc69c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.612+0000 7fc6a6000640 1 -- 172.21.15.67:0/3594395134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c03f070 con 0x7fc6a80754c0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.612+0000 7fc6a6000640 1 -- 172.21.15.67:0/3594395134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc69c03baf0 con 0x7fc6a80754c0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.612+0000 7fc6a6000640 1 -- 172.21.15.67:0/3594395134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c035030 con 0x7fc6a80754c0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.612+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81aba00 con 0x7fc6a80754c0 2024-09-10T23:06:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.612+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81abf30 con 0x7fc6a80754c0 2024-09-10T23:06:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.616+0000 7fc6b2de0740 1 Processor -- start 2024-09-10T23:06:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.616+0000 7fc6b2de0740 1 -- start start 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe652f0 0x557fdfe656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 0x557fdfe71660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557fdfddaca0 con 0x557fdfe65c10 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557fdfdca0d0 con 0x557fdfe652f0 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557fdfdcdee0 con 0x557fdfe71ba0 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6a6000640 1 -- 172.21.15.67:0/3594395134 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc69c03b660 con 0x7fc6a80754c0 2024-09-10T23:06:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6ace00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6ace00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57380/0 (socket says 172.21.15.67:57380) 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6ace00640 1 -- 172.21.15.67:0/3571406914 learned_addr learned my addr 172.21.15.67:0/3571406914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.617+0000 7fc6ae200640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe652f0 0x557fdfe656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ad800640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 0x557fdfe71660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ace00640 1 -- 172.21.15.67:0/3571406914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe652f0 msgr2=0x557fdfe656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ace00640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe652f0 0x557fdfe656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ace00640 1 -- 172.21.15.67:0/3571406914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 msgr2=0x557fdfe71660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ace00640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 0x557fdfe71660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ace00640 1 -- 172.21.15.67:0/3571406914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdfd7c400 con 0x557fdfe71ba0 2024-09-10T23:06:29.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.618+0000 7fc6ad800640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 0x557fdfe71660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:29.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.619+0000 7fc6ace00640 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc69400bcb0 tx=0x7fc6940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8828c841ba45bea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.619+0000 7fc69aa00640 1 -- 172.21.15.67:0/3571406914 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694004030 con 0x557fdfe71ba0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.619+0000 7fc69aa00640 1 -- 172.21.15.67:0/3571406914 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6940035f0 con 0x557fdfe71ba0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.619+0000 7fc69aa00640 1 -- 172.21.15.67:0/3571406914 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694011680 con 0x557fdfe71ba0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6a6000640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 0x7fc6900b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6a6000640 1 -- 172.21.15.67:0/3594395134 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7fc69c106cf0 con 0x7fc6a80754c0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6ae200640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 0x7fc6900b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 -- 172.21.15.67:0/3571406914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 msgr2=0x557fdfe73f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc69400bcb0 tx=0x7fc6940107a0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 -- 172.21.15.67:0/3571406914 shutdown_connections 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe73f90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe65c10 0x557fdfe71660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/3571406914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe652f0 0x557fdfe656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.620+0000 7fc6b2de0740 1 -- 172.21.15.67:0/3571406914 >> 172.21.15.67:0/3571406914 conn(0x557fdfe63120 msgr2=0x557fdfe642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.621+0000 7fc6b2de0740 1 -- 172.21.15.67:0/3571406914 shutdown_connections 2024-09-10T23:06:29.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.621+0000 7fc6b2de0740 1 -- 172.21.15.67:0/3571406914 wait complete. 2024-09-10T23:06:29.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.621+0000 7fc6ae200640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 0x7fc6900b7a40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806c220 tx=0x7fc6a0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.622+0000 7fc6b2de0740 1 -- start start 2024-09-10T23:06:29.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.622+0000 7fc6b2de0740 1 Processor -- start 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe6e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557fdfc2b7a0 con 0x557fdfe652f0 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557fdfd6fa10 con 0x557fdfe71ba0 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557fdfe74820 con 0x557fdfe65c10 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6b2de0740 10 client.0 _ll_register_callbacks cb 0x557fdfd7fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:29.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55038/0 (socket says 172.21.15.67:55038) 2024-09-10T23:06:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6ad800640 1 -- 172.21.15.67:0/1550966012 learned_addr learned my addr 172.21.15.67:0/1550966012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.623+0000 7fc6ae200640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ace00640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe6e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 -- 172.21.15.67:0/1550966012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 msgr2=0x557fdfe6e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe6e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 -- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 msgr2=0x557fdfe6ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe6ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 -- 172.21.15.67:0/1550966012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdfb37b90 con 0x557fdfe652f0 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ace00640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe6e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:29.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc6ad800640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0ca000 tx=0x7fc69c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1810401945b5452 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.624+0000 7fc68f400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c108030 con 0x557fdfe652f0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7fc69c0c9000 con 0x557fdfe652f0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6940035f0 con 0x557fdfe652f0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc69c049070 con 0x557fdfe652f0 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc6b2de0740 10 client.15903 fetch_fsmap learned FSMap version 31 2024-09-10T23:06:29.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.625+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557fdfe80150 con 0x557fdfe652f0 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6940035f0 con 0x557fdfe652f0 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc69c1042e0 con 0x557fdfe652f0 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 10 client.15903 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 10 client.15903 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 20 client.15903 populate_metadata read hostname 'smithi067' 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfe816a0 0x557fdfe83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 4 client.15903 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.626+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fdfddcbf0 con 0x557fdfe816a0 2024-09-10T23:06:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.627+0000 7fc6ace00640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfe816a0 0x557fdfe83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.628+0000 7fc6ace00640 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfe816a0 0x557fdfe83b40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589826ef77e93134 server_cookie=f2e676b0a55e6bcf in_seq=0 out_seq=0 2024-09-10T23:06:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.628+0000 7fc68f400640 10 client.15903 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.628+0000 7fc68f400640 1 -- 172.21.15.67:0/1550966012 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fdfddcbf0 con 0x557fdfe816a0 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.628+0000 7fc68f400640 10 client.15903 handle_command_reply: tid=0 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 client.15903 shutdown 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 msgr2=0x557fdfe71460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0ca000 tx=0x7fc69c02a410 comp rx=0 tx=0).stop 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 shutdown_connections 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfe816a0 0x557fdfe83b40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe6ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe6e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/1550966012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe71460 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc68f400640 0 client.15903 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 >> 172.21.15.67:0/1550966012 conn(0x557fdfe63120 msgr2=0x557fdfe68cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 shutdown_connections 2024-09-10T23:06:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.629+0000 7fc6b2de0740 1 -- 172.21.15.67:0/1550966012 wait complete. 2024-09-10T23:06:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.630+0000 7fc6b2de0740 20 client.15903 trim_cache size 0 max 16384 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 Processor -- start 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 -- start start 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe65c10 0x557fdfe6bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b8ae0 con 0x557fdfe652f0 2024-09-10T23:06:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b63b0 con 0x557fdfe65c10 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.652+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b4220 con 0x557fdfe71ba0 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ace00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe65c10 0x557fdfe6bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55050/0 (socket says 172.21.15.67:55050) 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 -- 172.21.15.67:0/295552713 learned_addr learned my addr 172.21.15.67:0/295552713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ae200640 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 -- 172.21.15.67:0/295552713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 msgr2=0x557fdfe6c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe6c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 -- 172.21.15.67:0/295552713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe65c10 msgr2=0x557fdfe6bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe65c10 0x557fdfe6bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.653+0000 7fc6ad800640 1 -- 172.21.15.67:0/295552713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdfe69600 con 0x557fdfe652f0 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.654+0000 7fc6ad800640 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02a9d0 tx=0x7fc69c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f1f189b28cea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.654+0000 7fc69a000640 1 -- 172.21.15.67:0/295552713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c07c030 con 0x557fdfe652f0 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.654+0000 7fc69a000640 1 -- 172.21.15.67:0/295552713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc69c0cc8a0 con 0x557fdfe652f0 2024-09-10T23:06:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.654+0000 7fc69a000640 1 -- 172.21.15.67:0/295552713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0cc500 con 0x557fdfe652f0 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 -- 172.21.15.67:0/295552713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 msgr2=0x557fdfe621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02a9d0 tx=0x7fc69c009260 comp rx=0 tx=0).stop 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 -- 172.21.15.67:0/295552713 shutdown_connections 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe71ba0 0x557fdfe6c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe65c10 0x557fdfe6bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/295552713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe621f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 -- 172.21.15.67:0/295552713 >> 172.21.15.67:0/295552713 conn(0x557fdfe63120 msgr2=0x557fdfe61ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 -- 172.21.15.67:0/295552713 shutdown_connections 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.655+0000 7fc6b2de0740 1 -- 172.21.15.67:0/295552713 wait complete. 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.656+0000 7fc6b2de0740 1 -- start start 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.656+0000 7fc6b2de0740 1 Processor -- start 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.656+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b4870 con 0x557fdfe652f0 2024-09-10T23:06:29.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b5ce0 con 0x557fdfe71ba0 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b4f40 con 0x557fdfe65c10 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6b2de0740 10 client.0 _ll_register_callbacks cb 0x7fc69c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6ace00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6ace00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57400/0 (socket says 172.21.15.67:57400) 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.657+0000 7fc6ace00640 1 -- 172.21.15.67:0/2526247931 learned_addr learned my addr 172.21.15.67:0/2526247931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 -- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 msgr2=0x557fdfe775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 -- 172.21.15.67:0/2526247931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 msgr2=0x557fdfe76b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe76b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 -- 172.21.15.67:0/2526247931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdfeee790 con 0x557fdfe65c10 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6ace00640 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc69400bc80 tx=0x7fc69400bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b1660e2c825a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc68f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694003a00 con 0x557fdfe65c10 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc69c0cc8a0 con 0x557fdfe65c10 2024-09-10T23:06:29.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.658+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7fc694005040 con 0x557fdfe65c10 2024-09-10T23:06:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.663+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc69403b050 con 0x557fdfe65c10 2024-09-10T23:06:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.663+0000 7fc6b2de0740 10 client.25633 fetch_fsmap learned FSMap version 31 2024-09-10T23:06:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.663+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557fdfeefa70 con 0x557fdfe65c10 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc69c0cc8a0 con 0x557fdfe65c10 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc69402d7e0 con 0x557fdfe65c10 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 10 client.25633 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 10 client.25633 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 20 client.25633 populate_metadata read hostname 'smithi067' 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfeeffd0 0x557fdfef23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 4 client.25633 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-09-10T23:06:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.664+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x557fdfdd86a0 con 0x557fdfeeffd0 2024-09-10T23:06:29.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.665+0000 7fc6ad800640 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfeeffd0 0x557fdfef23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:29.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.665+0000 7fc6ad800640 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfeeffd0 0x557fdfef23f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee536e7c95ab6c16 server_cookie=90c04eca3feac60f in_seq=0 out_seq=0 2024-09-10T23:06:29.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.665+0000 7fc68f400640 10 client.25633 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc68f400640 1 -- 172.21.15.67:0/2526247931 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557fdfdd86a0 con 0x557fdfeeffd0 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc68f400640 10 client.25633 handle_command_reply: tid=0 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 client.25633 shutdown 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 msgr2=0x557fdfe77070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc69400bc80 tx=0x7fc69400bbf0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 shutdown_connections 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557fdfeeffd0 0x557fdfef23f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557fdfe71ba0 0x557fdfe775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc68f400640 0 client.25633 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557fdfe65c10 0x557fdfe77070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 --2- 172.21.15.67:0/2526247931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557fdfe652f0 0x557fdfe76b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 >> 172.21.15.67:0/2526247931 conn(0x557fdfe63120 msgr2=0x557fdfe70700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.789+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 shutdown_connections 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6b2de0740 1 -- 172.21.15.67:0/2526247931 wait complete. 2024-09-10T23:06:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6b2de0740 20 client.25633 trim_cache size 0 max 16384 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 22, 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "8b713d0a": { 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 22, 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:06:29.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 146.8, 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:29.794 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 msgr2=0x7fc6900b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 0x7fc6900b7a40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6a806c220 tx=0x7fc6a0009500 comp rx=0 tx=0).stop 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 msgr2=0x7fc6a8102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.790+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02a930 tx=0x7fc69c003b90 comp rx=0 tx=0).stop 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 shutdown_connections 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc6900b5580 0x7fc6900b7a40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6a810f210 0x7fc6a8103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a8075de0 0x7fc6a8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 --2- 172.21.15.67:0/3594395134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a80754c0 0x7fc6a8102980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 >> 172.21.15.67:0/3594395134 conn(0x7fc6a80fe6d0 msgr2=0x7fc6a80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 shutdown_connections 2024-09-10T23:06:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:29.791+0000 7fc6aec00640 1 -- 172.21.15.67:0/3594395134 wait complete. 2024-09-10T23:06:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:29 smithi092 ceph-mon[34223]: pgmap v617: 129 pgs: 129 active+clean; 3.2 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:06:30.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:29 smithi067 ceph-mon[30939]: pgmap v617: 129 pgs: 129 active+clean; 3.2 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:06:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:29 smithi145 ceph-mon[34397]: pgmap v617: 129 pgs: 129 active+clean; 3.2 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:06:30.737 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8b713d0a', quiesce took 4.8 seconds. Will release after: 22.0 2024-09-10T23:06:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:31 smithi092 ceph-mon[34223]: pgmap v618: 129 pgs: 129 active+clean; 3.3 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:31 smithi067 ceph-mon[30939]: pgmap v618: 129 pgs: 129 active+clean; 3.3 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:31 smithi145 ceph-mon[34397]: pgmap v618: 129 pgs: 129 active+clean; 3.3 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 15 op/s 2024-09-10T23:06:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:33 smithi092 ceph-mon[34223]: pgmap v619: 129 pgs: 129 active+clean; 3.3 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:06:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:33 smithi067 ceph-mon[30939]: pgmap v619: 129 pgs: 129 active+clean; 3.3 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:06:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:33 smithi145 ceph-mon[34397]: pgmap v619: 129 pgs: 129 active+clean; 3.3 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:06:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:35 smithi092 ceph-mon[34223]: pgmap v620: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:06:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:35 smithi067 ceph-mon[30939]: pgmap v620: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:06:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:35 smithi145 ceph-mon[34397]: pgmap v620: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:06:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:37 smithi092 ceph-mon[34223]: pgmap v621: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 11 op/s 2024-09-10T23:06:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:37 smithi067 ceph-mon[30939]: pgmap v621: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 11 op/s 2024-09-10T23:06:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:37 smithi145 ceph-mon[34397]: pgmap v621: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 11 op/s 2024-09-10T23:06:39.932 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:39 smithi145 ceph-mon[34397]: pgmap v622: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:06:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:39 smithi092 ceph-mon[34223]: pgmap v622: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:06:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:39 smithi067 ceph-mon[30939]: pgmap v622: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 8 op/s 2024-09-10T23:06:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:41 smithi092 ceph-mon[34223]: pgmap v623: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:06:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:41 smithi067 ceph-mon[30939]: pgmap v623: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:06:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:41 smithi145 ceph-mon[34397]: pgmap v623: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:06:43.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:42 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:43.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:42 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:43.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:42 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:43 smithi092 ceph-mon[34223]: pgmap v624: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-09-10T23:06:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:06:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:43 smithi067 ceph-mon[30939]: pgmap v624: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-09-10T23:06:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:06:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:43 smithi145 ceph-mon[34397]: pgmap v624: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-09-10T23:06:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:06:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:45 smithi092 ceph-mon[34223]: pgmap v625: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:06:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:45 smithi067 ceph-mon[30939]: pgmap v625: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:06:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:45 smithi145 ceph-mon[34397]: pgmap v625: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:06:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:47 smithi092 ceph-mon[34223]: pgmap v626: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 422 KiB/s wr, 7 op/s 2024-09-10T23:06:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:47 smithi067 ceph-mon[30939]: pgmap v626: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 422 KiB/s wr, 7 op/s 2024-09-10T23:06:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:47 smithi145 ceph-mon[34397]: pgmap v626: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 422 KiB/s wr, 7 op/s 2024-09-10T23:06:49.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:49 smithi067 ceph-mon[30939]: pgmap v627: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 420 KiB/s wr, 5 op/s 2024-09-10T23:06:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:49 smithi092 ceph-mon[34223]: pgmap v627: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 420 KiB/s wr, 5 op/s 2024-09-10T23:06:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:49 smithi145 ceph-mon[34397]: pgmap v627: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 420 KiB/s wr, 5 op/s 2024-09-10T23:06:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:51 smithi092 ceph-mon[34223]: pgmap v628: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 420 KiB/s wr, 6 op/s 2024-09-10T23:06:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:51 smithi067 ceph-mon[30939]: pgmap v628: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 420 KiB/s wr, 6 op/s 2024-09-10T23:06:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:51 smithi145 ceph-mon[34397]: pgmap v628: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 420 KiB/s wr, 6 op/s 2024-09-10T23:06:52.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:06:53.130 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 -- 172.21.15.67:0/3257325183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174104df0 msgr2=0x7f01741051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 --2- 172.21.15.67:0/3257325183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174104df0 0x7f01741051d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0168009900 tx=0x7f016802a400 comp rx=0 tx=0).stop 2024-09-10T23:06:53.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 -- 172.21.15.67:0/3257325183 shutdown_connections 2024-09-10T23:06:53.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 --2- 172.21.15.67:0/3257325183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017410b180 0x7f0174111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 --2- 172.21.15.67:0/3257325183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 0x7f017410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 --2- 172.21.15.67:0/3257325183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174104df0 0x7f01741051d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 -- 172.21.15.67:0/3257325183 >> 172.21.15.67:0/3257325183 conn(0x7f0174100840 msgr2=0x7f0174102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 -- 172.21.15.67:0/3257325183 shutdown_connections 2024-09-10T23:06:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.723+0000 7f017a600640 1 -- 172.21.15.67:0/3257325183 wait complete. 2024-09-10T23:06:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 Processor -- start 2024-09-10T23:06:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 -- start start 2024-09-10T23:06:53.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0174104df0 0x7f017419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 0x7f017419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0174114d40 con 0x7f017410b180 2024-09-10T23:06:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0174114bc0 con 0x7f017410a800 2024-09-10T23:06:53.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.724+0000 7f017a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0174114ec0 con 0x7f0174104df0 2024-09-10T23:06:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53890/0 (socket says 172.21.15.67:53890) 2024-09-10T23:06:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 -- 172.21.15.67:0/130203526 learned_addr learned my addr 172.21.15.67:0/130203526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0173e00640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 0x7f017419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:53.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179200640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0174104df0 0x7f017419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 -- 172.21.15.67:0/130203526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0174104df0 msgr2=0x7f017419b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0174104df0 0x7f017419b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 -- 172.21.15.67:0/130203526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 msgr2=0x7f017419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 0x7f017419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.725+0000 7f0179c00640 1 -- 172.21.15.67:0/130203526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741a11f0 con 0x7f017410b180 2024-09-10T23:06:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f0179c00640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f016c00eaa0 tx=0x7f016c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c00cdb0 con 0x7f017410b180 2024-09-10T23:06:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016c005020 con 0x7f017410b180 2024-09-10T23:06:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0116d0 con 0x7f017410b180 2024-09-10T23:06:53.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741a14e0 con 0x7f017410b180 2024-09-10T23:06:53.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.726+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a7ba0 con 0x7f017410b180 2024-09-10T23:06:53.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.727+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01741051d0 con 0x7f017410b180 2024-09-10T23:06:53.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.729+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f016c022060 con 0x7f017410b180 2024-09-10T23:06:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.731+0000 7f0171600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 0x7f015c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.731+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7f016c0e6170 con 0x7f017410b180 2024-09-10T23:06:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.731+0000 7f0179200640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 0x7f015c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.732+0000 7f0179200640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 0x7f015c0b7a40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0168006010 tx=0x7f0168005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.733+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f016c016030 con 0x7f017410b180 2024-09-10T23:06:53.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.968+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01741a7f50 con 0x7f017410b180 2024-09-10T23:06:53.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:53 smithi067 ceph-mon[30939]: pgmap v629: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:06:53.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.969+0000 7f0171600640 1 -- 172.21.15.67:0/130203526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f016c0ac2e0 con 0x7f017410b180 2024-09-10T23:06:53.972 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:06:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:06:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:06:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 msgr2=0x7f015c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 0x7f015c0b7a40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0168006010 tx=0x7f0168005e80 comp rx=0 tx=0).stop 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 msgr2=0x7f01741a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f016c00eaa0 tx=0x7f016c00ef60 comp rx=0 tx=0).stop 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 shutdown_connections 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f015c0b5580 0x7f015c0b7a40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017410b180 0x7f01741a0a20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017410a800 0x7f017419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 --2- 172.21.15.67:0/130203526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0174104df0 0x7f017419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:53.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 >> 172.21.15.67:0/130203526 conn(0x7f0174100840 msgr2=0x7f0174101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:53.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 shutdown_connections 2024-09-10T23:06:53.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:53.971+0000 7f017a600640 1 -- 172.21.15.67:0/130203526 wait complete. 2024-09-10T23:06:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:53 smithi092 ceph-mon[34223]: pgmap v629: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:06:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:53 smithi145 ceph-mon[34397]: pgmap v629: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:06:54.774 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 8b713d0a --release --await' 2024-09-10T23:06:54.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 8b713d0a --release --await 2024-09-10T23:06:55.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:54 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/130203526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:55.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:54 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/130203526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:55.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:54 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/130203526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:55.153 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 -- 172.21.15.67:0/3184636766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 msgr2=0x7f15e8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 --2- 172.21.15.67:0/3184636766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e8075920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009900 tx=0x7f15d802a410 comp rx=0 tx=0).stop 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 -- 172.21.15.67:0/3184636766 shutdown_connections 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 --2- 172.21.15.67:0/3184636766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 0x7f15e81117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 --2- 172.21.15.67:0/3184636766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 0x7f15e810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 --2- 172.21.15.67:0/3184636766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e8075920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 -- 172.21.15.67:0/3184636766 >> 172.21.15.67:0/3184636766 conn(0x7f15e80fe830 msgr2=0x7f15e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 -- 172.21.15.67:0/3184636766 shutdown_connections 2024-09-10T23:06:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.765+0000 7f15eda00640 1 -- 172.21.15.67:0/3184636766 wait complete. 2024-09-10T23:06:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 Processor -- start 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 -- start start 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 0x7f15e819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 0x7f15e81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8114e40 con 0x7f15e8075540 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8114cc0 con 0x7f15e810f3e0 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.766+0000 7f15eda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8114fc0 con 0x7f15e8075f40 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53906/0 (socket says 172.21.15.67:53906) 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 -- 172.21.15.67:0/2965165268 learned_addr learned my addr 172.21.15.67:0/2965165268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e7400640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 0x7f15e81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15de000640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 0x7f15e819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 -- 172.21.15.67:0/2965165268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 msgr2=0x7f15e819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 0x7f15e819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 -- 172.21.15.67:0/2965165268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 msgr2=0x7f15e81a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 0x7f15e81a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 -- 172.21.15.67:0/2965165268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81a1500 con 0x7f15e8075540 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.767+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009230 tx=0x7f15d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.768+0000 7f15dfe00640 1 -- 172.21.15.67:0/2965165268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d803f030 con 0x7f15e8075540 2024-09-10T23:06:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.768+0000 7f15dfe00640 1 -- 172.21.15.67:0/2965165268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15d8003a80 con 0x7f15e8075540 2024-09-10T23:06:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.768+0000 7f15dfe00640 1 -- 172.21.15.67:0/2965165268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8035070 con 0x7f15e8075540 2024-09-10T23:06:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.768+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81a1790 con 0x7f15e8075540 2024-09-10T23:06:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.768+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a7eb0 con 0x7f15e8075540 2024-09-10T23:06:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 Processor -- start 2024-09-10T23:06:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 -- start start 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15dfe00640 1 -- 172.21.15.67:0/2965165268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f15d803bc30 con 0x7f15e8075540 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 0x560e5dfc4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e5df2f290 con 0x560e5dfc48a0 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e5df1e6c0 con 0x560e5dfb7ff0 2024-09-10T23:06:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.771+0000 7f15f0614740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e5df224d0 con 0x560e5dfb8910 2024-09-10T23:06:55.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:45364/0 (socket says 172.21.15.67:45364) 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 -- 172.21.15.67:0/2571317483 learned_addr learned my addr 172.21.15.67:0/2571317483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 0x560e5dfc4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 -- 172.21.15.67:0/2571317483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 msgr2=0x560e5dfc4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 0x560e5dfc4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 -- 172.21.15.67:0/2571317483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 msgr2=0x560e5dfc6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e7400640 1 -- 172.21.15.67:0/2571317483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e5deab460 con 0x560e5dfb7ff0 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15de000640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.772+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 0x560e5dfc4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15e7400640 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009fc0 tx=0x7f15e0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9430d2e9d20d1931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15d7e00640 1 -- 172.21.15.67:0/2571317483 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0016070 con 0x560e5dfb7ff0 2024-09-10T23:06:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15d7e00640 1 -- 172.21.15.67:0/2571317483 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15e0004d20 con 0x560e5dfb7ff0 2024-09-10T23:06:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15d7e00640 1 -- 172.21.15.67:0/2571317483 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005280 con 0x560e5dfb7ff0 2024-09-10T23:06:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15dfe00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 0x7f15cc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15dfe00640 1 -- 172.21.15.67:0/2965165268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7f15d8106cf0 con 0x7f15e8075540 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.773+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 0x7f15cc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 -- 172.21.15.67:0/2571317483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 msgr2=0x560e5dfb83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009fc0 tx=0x7f15e0010800 comp rx=0 tx=0).stop 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 -- 172.21.15.67:0/2571317483 shutdown_connections 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb8910 0x560e5dfc4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 --2- 172.21.15.67:0/2571317483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb83d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 -- 172.21.15.67:0/2571317483 >> 172.21.15.67:0/2571317483 conn(0x560e5dfb5e20 msgr2=0x560e5dfb6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 -- 172.21.15.67:0/2571317483 shutdown_connections 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15e6a00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 0x7f15cc0b7990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560e5dfbb170 tx=0x7f15d8009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.774+0000 7f15f0614740 1 -- 172.21.15.67:0/2571317483 wait complete. 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 -- start start 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 Processor -- start 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfc1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfc1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x560e5dd7fbb0 con 0x560e5dfc48a0 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e5deb96c0 con 0x560e5dfb8910 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560e5dfc7550 con 0x560e5dfb7ff0 2024-09-10T23:06:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.775+0000 7f15f0614740 10 client.0 _ll_register_callbacks cb 0x560e5ded4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfc1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53920/0 (socket says 172.21.15.67:53920) 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15e7400640 1 -- 172.21.15.67:0/4120376068 learned_addr learned my addr 172.21.15.67:0/4120376068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15e7400640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfc1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 -- 172.21.15.67:0/4120376068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 msgr2=0x560e5dfc1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfc1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 -- 172.21.15.67:0/4120376068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 msgr2=0x560e5dfc2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15de000640 1 -- 172.21.15.67:0/4120376068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5dc7cb00 con 0x560e5dfb8910 2024-09-10T23:06:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.776+0000 7f15e7400640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15de000640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfc1dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f15e8111b10 tx=0x7f15d000bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df46a41bb06dd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0003df0 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e0004d20 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7f15d0005020 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15d0005220 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15f0614740 10 client.35645 fetch_fsmap learned FSMap version 31 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e5dcf3960 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.777+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15e0004d20 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f15d001b470 con 0x560e5dfb8910 2024-09-10T23:06:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 10 client.35645 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 10 client.35645 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 20 client.35645 populate_metadata read hostname 'smithi067' 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5dfd4500 0x560e5dfd6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 4 client.35645 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e5df311e0 con 0x560e5dfd4500 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.778+0000 7f15e7400640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5dfd4500 0x560e5dfd6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.779+0000 7f15e7400640 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5dfd4500 0x560e5dfd6920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c6a63d9281441b server_cookie=ea97261aa83455ed in_seq=0 out_seq=0 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.779+0000 7f15d4c00640 10 client.35645 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15d4c00640 1 -- 172.21.15.67:0/4120376068 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e5df311e0 con 0x560e5dfd4500 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15d4c00640 10 client.35645 handle_command_reply: tid=0 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 client.35645 shutdown 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 msgr2=0x560e5dfc1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfc1dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f15e8111b10 tx=0x7f15d000bd80 comp rx=0 tx=0).stop 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 shutdown_connections 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5dfd4500 0x560e5dfd6920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfc1dd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 --2- 172.21.15.67:0/4120376068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfc1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 >> 172.21.15.67:0/4120376068 conn(0x560e5dfb5e20 msgr2=0x560e5dfbb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15d4c00640 0 client.35645 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 shutdown_connections 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.780+0000 7f15f0614740 1 -- 172.21.15.67:0/4120376068 wait complete. 2024-09-10T23:06:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.781+0000 7f15f0614740 20 client.35645 trim_cache size 0 max 16384 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.798+0000 7f15f0614740 1 Processor -- start 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.798+0000 7f15f0614740 1 -- start start 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfc48a0 0x560e5dfdc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b49f0 con 0x560e5dfb8910 2024-09-10T23:06:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b22c0 con 0x560e5dfc48a0 2024-09-10T23:06:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15f0614740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b0130 con 0x560e5dfb7ff0 2024-09-10T23:06:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53924/0 (socket says 172.21.15.67:53924) 2024-09-10T23:06:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 -- 172.21.15.67:0/2869379573 learned_addr learned my addr 172.21.15.67:0/2869379573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15e7400640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfc48a0 0x560e5dfdc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15e7400640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 -- 172.21.15.67:0/2869379573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 msgr2=0x560e5dfb4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 -- 172.21.15.67:0/2869379573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfc48a0 msgr2=0x560e5dfdc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfc48a0 0x560e5dfdc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.799+0000 7f15de000640 1 -- 172.21.15.67:0/2869379573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e5dfb68a0 con 0x560e5dfb8910 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.800+0000 7f15e7400640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.800+0000 7f15de000640 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15d001b910 tx=0x7f15d001cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d3f7c0fb0f1389 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.800+0000 7f15d7400640 1 -- 172.21.15.67:0/2869379573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d002c070 con 0x560e5dfb8910 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.800+0000 7f15d7400640 1 -- 172.21.15.67:0/2869379573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15d000b400 con 0x560e5dfb8910 2024-09-10T23:06:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.800+0000 7f15d7400640 1 -- 172.21.15.67:0/2869379573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0005600 con 0x560e5dfb8910 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- 172.21.15.67:0/2869379573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 msgr2=0x560e5dfdc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15d001b910 tx=0x7f15d001cf60 comp rx=0 tx=0).stop 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- 172.21.15.67:0/2869379573 shutdown_connections 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfc48a0 0x560e5dfdc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfb8910 0x560e5dfdc1b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 --2- 172.21.15.67:0/2869379573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- 172.21.15.67:0/2869379573 >> 172.21.15.67:0/2869379573 conn(0x560e5dfb5e20 msgr2=0x560e5dfb4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- 172.21.15.67:0/2869379573 shutdown_connections 2024-09-10T23:06:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- 172.21.15.67:0/2869379573 wait complete. 2024-09-10T23:06:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.801+0000 7f15f0614740 1 -- start start 2024-09-10T23:06:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 Processor -- start 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfcc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b0780 con 0x560e5dfc48a0 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b1bf0 con 0x560e5dfb8910 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b0e50 con 0x560e5dfb7ff0 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15f0614740 10 client.0 _ll_register_callbacks cb 0x7f15d0004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53940/0 (socket says 172.21.15.67:53940) 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15e7400640 1 -- 172.21.15.67:0/1528298157 learned_addr learned my addr 172.21.15.67:0/1528298157 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.802+0000 7f15de000640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfcc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 -- 172.21.15.67:0/1528298157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 msgr2=0x560e5dfcc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfcc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 -- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 msgr2=0x560e5dfccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 -- 172.21.15.67:0/1528298157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5dfc7d10 con 0x560e5dfc48a0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15de000640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15e7400640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f90 tx=0x7f15d0004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b32d433bf49051e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.803+0000 7f15d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0016040 con 0x560e5dfc48a0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9519+0+0 (secure 0 0 0) 0x7f15e0012940 con 0x560e5dfc48a0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d000b400 con 0x560e5dfc48a0 2024-09-10T23:06:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15e0003970 con 0x560e5dfc48a0 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15f0614740 10 client.15933 fetch_fsmap learned FSMap version 31 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e5e041ba0 con 0x560e5dfc48a0 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15d000b400 con 0x560e5dfc48a0 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f15e002a070 con 0x560e5dfc48a0 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.804+0000 7f15f0614740 10 client.15933 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15f0614740 10 client.15933 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15f0614740 20 client.15933 populate_metadata read hostname 'smithi067' 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5e042070 0x560e5e044490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15f0614740 4 client.15933 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8b713d0a", "release": true, "await": true}] 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8b713d0a", "release": true, "await": true}) -- 0x560e5df2cc90 con 0x560e5e042070 2024-09-10T23:06:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.805+0000 7f15de000640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5e042070 0x560e5e044490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.806+0000 7f15de000640 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5e042070 0x560e5e044490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e6f952a65afa1e server_cookie=59dece97fe20e2b8 in_seq=0 out_seq=0 2024-09-10T23:06:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.806+0000 7f15d4c00640 10 client.15933 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.830+0000 7f15d4c00640 1 -- 172.21.15.67:0/1528298157 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e5df2cc90 con 0x560e5e042070 2024-09-10T23:06:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.830+0000 7f15d4c00640 10 client.15933 handle_command_reply: tid=0 2024-09-10T23:06:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.830+0000 7f15f0614740 1 client.15933 shutdown 2024-09-10T23:06:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.830+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 msgr2=0x560e5dfc77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.830+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f90 tx=0x7f15d0004490 comp rx=0 tx=0).stop 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 shutdown_connections 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x560e5e042070 0x560e5e044490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560e5dfc48a0 0x560e5dfc77d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e5dfb8910 0x560e5dfccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 --2- 172.21.15.67:0/1528298157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e5dfb7ff0 0x560e5dfcc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15d4c00640 0 client.15933 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 >> 172.21.15.67:0/1528298157 conn(0x560e5dfb5e20 msgr2=0x560e5dfc0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 shutdown_connections 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 1 -- 172.21.15.67:0/1528298157 wait complete. 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.831+0000 7f15f0614740 20 client.15933 trim_cache size 0 max 16384 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 24, 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "8b713d0a": { 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 24, 2024-09-10T23:06:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 146.8, 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 msgr2=0x7f15cc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 0x7f15cc0b7990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560e5dfbb170 tx=0x7f15d8009260 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 msgr2=0x7f15e819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009230 tx=0x7f15d8004270 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 shutdown_connections 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f15cc0b54d0 0x7f15cc0b7990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15e810f3e0 0x7f15e81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15e8075f40 0x7f15e819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 --2- 172.21.15.67:0/2965165268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15e8075540 0x7f15e819b450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 >> 172.21.15.67:0/2965165268 conn(0x7f15e80fe830 msgr2=0x7f15e80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 shutdown_connections 2024-09-10T23:06:55.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:06:55.832+0000 7f15eda00640 1 -- 172.21.15.67:0/2965165268 wait complete. 2024-09-10T23:06:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:55 smithi092 ceph-mon[34223]: pgmap v630: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:06:56.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:55 smithi067 ceph-mon[30939]: pgmap v630: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:06:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:55 smithi145 ceph-mon[34397]: pgmap v630: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:06:56.701 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8b713d0a', total seconds elapsed: 30.7 2024-09-10T23:06:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:57 smithi092 ceph-mon[34223]: pgmap v631: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:06:58.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:57 smithi067 ceph-mon[30939]: pgmap v631: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:06:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:57 smithi145 ceph-mon[34397]: pgmap v631: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-10T23:07:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:06:59 smithi092 ceph-mon[34223]: pgmap v632: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:07:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:06:59 smithi067 ceph-mon[30939]: pgmap v632: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:07:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:06:59 smithi145 ceph-mon[34397]: pgmap v632: 129 pgs: 129 active+clean; 3.3 GiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:07:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:01 smithi092 ceph-mon[34223]: pgmap v633: 129 pgs: 129 active+clean; 3.3 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:07:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:01 smithi067 ceph-mon[30939]: pgmap v633: 129 pgs: 129 active+clean; 3.3 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:07:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:01 smithi145 ceph-mon[34397]: pgmap v633: 129 pgs: 129 active+clean; 3.3 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 MiB/s wr, 7 op/s 2024-09-10T23:07:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:03 smithi092 ceph-mon[34223]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-09-10T23:07:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:03 smithi067 ceph-mon[30939]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-09-10T23:07:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:03 smithi145 ceph-mon[34397]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-09-10T23:07:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:05 smithi092 ceph-mon[34223]: pgmap v635: 129 pgs: 129 active+clean; 3.4 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-09-10T23:07:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:05 smithi067 ceph-mon[30939]: pgmap v635: 129 pgs: 129 active+clean; 3.4 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-09-10T23:07:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:05 smithi145 ceph-mon[34397]: pgmap v635: 129 pgs: 129 active+clean; 3.4 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-09-10T23:07:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:07 smithi092 ceph-mon[34223]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 28 op/s 2024-09-10T23:07:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:07 smithi067 ceph-mon[30939]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 28 op/s 2024-09-10T23:07:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:07 smithi145 ceph-mon[34397]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 28 op/s 2024-09-10T23:07:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:08 smithi092 ceph-mon[34223]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T23:07:09.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:08 smithi145 ceph-mon[34397]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T23:07:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:08 smithi067 ceph-mon[30939]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T23:07:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:09 smithi092 ceph-mon[34223]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:07:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:09 smithi067 ceph-mon[30939]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:07:10.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:09 smithi145 ceph-mon[34397]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:07:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:11 smithi092 ceph-mon[34223]: pgmap v639: 129 pgs: 129 active+clean; 3.5 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:07:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:11 smithi067 ceph-mon[30939]: pgmap v639: 129 pgs: 129 active+clean; 3.5 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:07:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:11 smithi145 ceph-mon[34397]: pgmap v639: 129 pgs: 129 active+clean; 3.5 GiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:07:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:13 smithi092 ceph-mon[34223]: pgmap v640: 129 pgs: 129 active+clean; 3.5 GiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:07:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:13 smithi092 ceph-mon[34223]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T23:07:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:13 smithi067 ceph-mon[30939]: pgmap v640: 129 pgs: 129 active+clean; 3.5 GiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:07:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:13 smithi067 ceph-mon[30939]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T23:07:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:13 smithi145 ceph-mon[34397]: pgmap v640: 129 pgs: 129 active+clean; 3.5 GiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:07:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:13 smithi145 ceph-mon[34397]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T23:07:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:15 smithi092 ceph-mon[34223]: pgmap v642: 129 pgs: 129 active+clean; 3.5 GiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:15 smithi067 ceph-mon[30939]: pgmap v642: 129 pgs: 129 active+clean; 3.5 GiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:15 smithi145 ceph-mon[34397]: pgmap v642: 129 pgs: 129 active+clean; 3.5 GiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:17 smithi092 ceph-mon[34223]: pgmap v643: 129 pgs: 129 active+clean; 3.6 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:07:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:17 smithi067 ceph-mon[30939]: pgmap v643: 129 pgs: 129 active+clean; 3.6 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:07:18.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:17 smithi145 ceph-mon[34397]: pgmap v643: 129 pgs: 129 active+clean; 3.6 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:07:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:19 smithi092 ceph-mon[34223]: pgmap v644: 129 pgs: 129 active+clean; 3.6 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:07:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:19 smithi067 ceph-mon[30939]: pgmap v644: 129 pgs: 129 active+clean; 3.6 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:07:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:19 smithi145 ceph-mon[34397]: pgmap v644: 129 pgs: 129 active+clean; 3.6 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:07:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:21 smithi067 ceph-mon[30939]: pgmap v645: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:21 smithi145 ceph-mon[34397]: pgmap v645: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:21 smithi092 ceph-mon[34223]: pgmap v645: 129 pgs: 129 active+clean; 3.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:23 smithi067 ceph-mon[30939]: pgmap v646: 129 pgs: 129 active+clean; 3.7 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:23 smithi145 ceph-mon[34397]: pgmap v646: 129 pgs: 129 active+clean; 3.7 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:23 smithi092 ceph-mon[34223]: pgmap v646: 129 pgs: 129 active+clean; 3.7 GiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:25 smithi067 ceph-mon[30939]: pgmap v647: 129 pgs: 129 active+clean; 3.7 GiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:07:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:25 smithi145 ceph-mon[34397]: pgmap v647: 129 pgs: 129 active+clean; 3.7 GiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:07:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:25 smithi092 ceph-mon[34223]: pgmap v647: 129 pgs: 129 active+clean; 3.7 GiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:07:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:27 smithi067 ceph-mon[30939]: pgmap v648: 129 pgs: 129 active+clean; 3.7 GiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:27 smithi145 ceph-mon[34397]: pgmap v648: 129 pgs: 129 active+clean; 3.7 GiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:27 smithi092 ceph-mon[34223]: pgmap v648: 129 pgs: 129 active+clean; 3.7 GiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:29 smithi067 ceph-mon[30939]: pgmap v649: 129 pgs: 129 active+clean; 3.7 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:07:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:29 smithi145 ceph-mon[34397]: pgmap v649: 129 pgs: 129 active+clean; 3.7 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:07:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:29 smithi092 ceph-mon[34223]: pgmap v649: 129 pgs: 129 active+clean; 3.7 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:07:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:31 smithi067 ceph-mon[30939]: pgmap v650: 129 pgs: 129 active+clean; 3.8 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-09-10T23:07:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:31 smithi145 ceph-mon[34397]: pgmap v650: 129 pgs: 129 active+clean; 3.8 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-09-10T23:07:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:31 smithi092 ceph-mon[34223]: pgmap v650: 129 pgs: 129 active+clean; 3.8 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-09-10T23:07:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:33 smithi067 ceph-mon[30939]: pgmap v651: 129 pgs: 129 active+clean; 3.8 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:34.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:33 smithi145 ceph-mon[34397]: pgmap v651: 129 pgs: 129 active+clean; 3.8 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:33 smithi092 ceph-mon[34223]: pgmap v651: 129 pgs: 129 active+clean; 3.8 GiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:35 smithi067 ceph-mon[30939]: pgmap v652: 129 pgs: 129 active+clean; 3.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:07:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:35 smithi145 ceph-mon[34397]: pgmap v652: 129 pgs: 129 active+clean; 3.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:07:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:35 smithi092 ceph-mon[34223]: pgmap v652: 129 pgs: 129 active+clean; 3.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:07:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:37 smithi067 ceph-mon[30939]: pgmap v653: 129 pgs: 129 active+clean; 3.9 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:37 smithi145 ceph-mon[34397]: pgmap v653: 129 pgs: 129 active+clean; 3.9 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:37 smithi092 ceph-mon[34223]: pgmap v653: 129 pgs: 129 active+clean; 3.9 GiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:39 smithi145 ceph-mon[34397]: pgmap v654: 129 pgs: 129 active+clean; 3.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:39 smithi092 ceph-mon[34223]: pgmap v654: 129 pgs: 129 active+clean; 3.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:39 smithi067 ceph-mon[30939]: pgmap v654: 129 pgs: 129 active+clean; 3.9 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:07:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:41 smithi145 ceph-mon[34397]: pgmap v655: 129 pgs: 129 active+clean; 4.0 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:07:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:41 smithi092 ceph-mon[34223]: pgmap v655: 129 pgs: 129 active+clean; 4.0 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:07:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:41 smithi067 ceph-mon[30939]: pgmap v655: 129 pgs: 129 active+clean; 4.0 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:07:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:43 smithi145 ceph-mon[34397]: pgmap v656: 129 pgs: 129 active+clean; 4.0 GiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:43 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:43 smithi092 ceph-mon[34223]: pgmap v656: 129 pgs: 129 active+clean; 4.0 GiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:43 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:43 smithi067 ceph-mon[30939]: pgmap v656: 129 pgs: 129 active+clean; 4.0 GiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:43 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:45.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: pgmap v657: 129 pgs: 129 active+clean; 4.0 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:45.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:45.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:45.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: pgmap v657: 129 pgs: 129 active+clean; 4.0 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:45.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: pgmap v657: 129 pgs: 129 active+clean; 4.0 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:07:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:45.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:07:46.473 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.4 2024-09-10T23:07:46.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:07:46.949 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:47 smithi067 ceph-mon[30939]: pgmap v658: 129 pgs: 129 active+clean; 4.0 GiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 -- 172.21.15.67:0/39623834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 msgr2=0x7f86041119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 --2- 172.21.15.67:0/39623834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041119a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009ca0 tx=0x7f85fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 -- 172.21.15.67:0/39623834 shutdown_connections 2024-09-10T23:07:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 --2- 172.21.15.67:0/39623834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041119a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 --2- 172.21.15.67:0/39623834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 0x7f860410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 --2- 172.21.15.67:0/39623834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 0x7f86041051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 -- 172.21.15.67:0/39623834 >> 172.21.15.67:0/39623834 conn(0x7f8604100840 msgr2=0x7f8604102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 -- 172.21.15.67:0/39623834 shutdown_connections 2024-09-10T23:07:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.544+0000 7f860a400640 1 -- 172.21.15.67:0/39623834 wait complete. 2024-09-10T23:07:47.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.545+0000 7f860a400640 1 Processor -- start 2024-09-10T23:07:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.545+0000 7f860a400640 1 -- start start 2024-09-10T23:07:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 0x7f860419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 0x7f860419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8604114d60 con 0x7f860410b1a0 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8604114be0 con 0x7f8604104e10 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f860a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8604114ee0 con 0x7f860410a820 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f8609000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 0x7f860419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.546+0000 7f8609a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8609a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34930/0 (socket says 172.21.15.67:34930) 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8609a00640 1 -- 172.21.15.67:0/2743835570 learned_addr learned my addr 172.21.15.67:0/2743835570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8603e00640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 0x7f860419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8609a00640 1 -- 172.21.15.67:0/2743835570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 msgr2=0x7f860419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8609a00640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 0x7f860419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.547+0000 7f8609a00640 1 -- 172.21.15.67:0/2743835570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 msgr2=0x7f860419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8609a00640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 0x7f860419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8609a00640 1 -- 172.21.15.67:0/2743835570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86041a11e0 con 0x7f860410b1a0 2024-09-10T23:07:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8609a00640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005e30 tx=0x7f85fc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc032d10 con 0x7f860410b1a0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85fc032eb0 con 0x7f860410b1a0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc030b20 con 0x7f860410b1a0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.548+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86041a1470 con 0x7f860410b1a0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.549+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a7b90 con 0x7f860410b1a0 2024-09-10T23:07:47.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.550+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86041051f0 con 0x7f860410b1a0 2024-09-10T23:07:47.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.553+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f85fc030cc0 con 0x7f860410b1a0 2024-09-10T23:07:47.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.555+0000 7f8601600640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 0x7f85ec0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.555+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9519+0+0 (secure 0 0 0) 0x7f85fc1076f0 con 0x7f860410b1a0 2024-09-10T23:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.555+0000 7f8609000640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 0x7f85ec0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.556+0000 7f8609000640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 0x7f85ec0b7a40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009bd0 tx=0x7f85f8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.558+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f85fc0cde30 con 0x7f860410b1a0 2024-09-10T23:07:47.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:47 smithi145 ceph-mon[34397]: pgmap v658: 129 pgs: 129 active+clean; 4.0 GiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:47.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:47 smithi092 ceph-mon[34223]: pgmap v658: 129 pgs: 129 active+clean; 4.0 GiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:07:47.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.805+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f860402e5c0 con 0x7f860410b1a0 2024-09-10T23:07:47.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.806+0000 7f8601600640 1 -- 172.21.15.67:0/2743835570 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f85fc0cd730 con 0x7f860410b1a0 2024-09-10T23:07:47.807 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:07:47.807 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:07:47.807 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 msgr2=0x7f85ec0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 0x7f85ec0b7a40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009bd0 tx=0x7f85f8005f90 comp rx=0 tx=0).stop 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 msgr2=0x7f86041a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005e30 tx=0x7f85fc0093d0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 shutdown_connections 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85ec0b5580 0x7f85ec0b7a40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f860410b1a0 0x7f86041a0a10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f860410a820 0x7f860419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 --2- 172.21.15.67:0/2743835570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8604104e10 0x7f860419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 >> 172.21.15.67:0/2743835570 conn(0x7f8604100840 msgr2=0x7f8604106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 shutdown_connections 2024-09-10T23:07:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:47.809+0000 7f860a400640 1 -- 172.21.15.67:0/2743835570 wait complete. 2024-09-10T23:07:48.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2743835570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:48.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2743835570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:48.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2743835570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:48.908 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0' 2024-09-10T23:07:48.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0 2024-09-10T23:07:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:49.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:49 smithi145 ceph-mon[34397]: pgmap v659: 129 pgs: 129 active+clean; 4.0 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:07:49.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:49 smithi067 ceph-mon[30939]: pgmap v659: 129 pgs: 129 active+clean; 4.0 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:07:49.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:49 smithi092 ceph-mon[34223]: pgmap v659: 129 pgs: 129 active+clean; 4.0 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:07:50.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 -- 172.21.15.67:0/3888309164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 msgr2=0x7f3d8c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 --2- 172.21.15.67:0/3888309164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c101ee0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009900 tx=0x7f3d7c02a410 comp rx=0 tx=0).stop 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 -- 172.21.15.67:0/3888309164 shutdown_connections 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 --2- 172.21.15.67:0/3888309164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 --2- 172.21.15.67:0/3888309164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 --2- 172.21.15.67:0/3888309164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c101ee0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 -- 172.21.15.67:0/3888309164 >> 172.21.15.67:0/3888309164 conn(0x7f3d8c0fd5b0 msgr2=0x7f3d8c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 -- 172.21.15.67:0/3888309164 shutdown_connections 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.008+0000 7f3d90a00640 1 -- 172.21.15.67:0/3888309164 wait complete. 2024-09-10T23:07:50.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 Processor -- start 2024-09-10T23:07:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 -- start start 2024-09-10T23:07:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c111a50 con 0x7f3d8c101b00 2024-09-10T23:07:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1118d0 con 0x7f3d8c107510 2024-09-10T23:07:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.010+0000 7f3d90a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c111bd0 con 0x7f3d8c107e90 2024-09-10T23:07:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d89e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:60350/0 (socket says 172.21.15.67:60350) 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55366/0 (socket says 172.21.15.67:55366) 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d89e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59676/0 (socket says 172.21.15.67:59676) 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 -- 172.21.15.67:0/1317003867 learned_addr learned my addr 172.21.15.67:0/1317003867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 -- 172.21.15.67:0/1317003867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 msgr2=0x7f3d8c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 -- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 msgr2=0x7f3d8c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.011+0000 7f3d8b200640 1 -- 172.21.15.67:0/1317003867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c197750 con 0x7f3d8c107e90 2024-09-10T23:07:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.012+0000 7f3d8b200640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400eaa0 tx=0x7f3d8400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.012+0000 7f3d83400640 1 -- 172.21.15.67:0/1317003867 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400cdb0 con 0x7f3d8c107e90 2024-09-10T23:07:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.012+0000 7f3d83400640 1 -- 172.21.15.67:0/1317003867 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d84005020 con 0x7f3d8c107e90 2024-09-10T23:07:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.013+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c197a40 con 0x7f3d8c107e90 2024-09-10T23:07:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.013+0000 7f3d83400640 1 -- 172.21.15.67:0/1317003867 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840116d0 con 0x7f3d8c107e90 2024-09-10T23:07:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.013+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c1b48d0 con 0x7f3d8c107e90 2024-09-10T23:07:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.017+0000 7f3d9405f740 1 Processor -- start 2024-09-10T23:07:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.017+0000 7f3d83400640 1 -- 172.21.15.67:0/1317003867 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3d8400f040 con 0x7f3d8c107e90 2024-09-10T23:07:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.017+0000 7f3d9405f740 1 -- start start 2024-09-10T23:07:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaaee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaafaba0 0x55fafaafcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fafaa63ca0 con 0x55fafaaee2f0 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fafaa530d0 con 0x55fafaaeec10 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d9405f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fafaa56ee0 con 0x55fafaafaba0 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59682/0 (socket says 172.21.15.67:59682) 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d8a800640 1 -- 172.21.15.67:0/193109941 learned_addr learned my addr 172.21.15.67:0/193109941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.018+0000 7f3d89e00640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaafaba0 0x55fafaafcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d89e00640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaaee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d8a800640 1 -- 172.21.15.67:0/193109941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaafaba0 msgr2=0x55fafaafcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d8a800640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaafaba0 0x55fafaafcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d8a800640 1 -- 172.21.15.67:0/193109941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 msgr2=0x55fafaaee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d83400640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 0x7f3d700b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d8a800640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaaee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d8a800640 1 -- 172.21.15.67:0/193109941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafaa05400 con 0x55fafaaeec10 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d89e00640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaaee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.019+0000 7f3d83400640 1 -- 172.21.15.67:0/1317003867 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9519+0+0 (secure 0 0 0) 0x7f3d840e61a0 con 0x7f3d8c107e90 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.020+0000 7f3d89e00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 0x7f3d700b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.020+0000 7f3d8a800640 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009900 tx=0x7f3d7c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84230992a0090eee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.020+0000 7f3d77400640 1 -- 172.21.15.67:0/193109941 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c040070 con 0x55fafaaeec10 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.020+0000 7f3d77400640 1 -- 172.21.15.67:0/193109941 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d7c0054b0 con 0x55fafaaeec10 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.020+0000 7f3d77400640 1 -- 172.21.15.67:0/193109941 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c035040 con 0x55fafaaeec10 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 -- 172.21.15.67:0/193109941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 msgr2=0x55fafaafa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009900 tx=0x7f3d7c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d89e00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 0x7f3d700b7a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d780058c0 tx=0x7f3d780046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 -- 172.21.15.67:0/193109941 shutdown_connections 2024-09-10T23:07:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaafaba0 0x55fafaafcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaaeec10 0x55fafaafa660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 --2- 172.21.15.67:0/193109941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaaee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.021+0000 7f3d9405f740 1 -- 172.21.15.67:0/193109941 >> 172.21.15.67:0/193109941 conn(0x55fafaaec120 msgr2=0x55fafaaed2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.022+0000 7f3d9405f740 1 -- 172.21.15.67:0/193109941 shutdown_connections 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.022+0000 7f3d9405f740 1 -- 172.21.15.67:0/193109941 wait complete. 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.022+0000 7f3d9405f740 1 -- start start 2024-09-10T23:07:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 Processor -- start 2024-09-10T23:07:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaeec10 0x55fafaaf78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fafa8b47a0 con 0x55fafaaee2f0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fafa9f8a10 con 0x55fafaafaba0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fafaafd7c0 con 0x55fafaaeec10 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.023+0000 7f3d9405f740 10 client.0 _ll_register_callbacks cb 0x55fafaa08be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.024+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.024+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55384/0 (socket says 172.21.15.67:55384) 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.024+0000 7f3d8a800640 1 -- 172.21.15.67:0/3012752992 learned_addr learned my addr 172.21.15.67:0/3012752992 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.024+0000 7f3d89e00640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaeec10 0x55fafaaf78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.024+0000 7f3d8b200640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 -- 172.21.15.67:0/3012752992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaeec10 msgr2=0x55fafaaf78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaeec10 0x55fafaaf78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 -- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 msgr2=0x55fafaaf7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 -- 172.21.15.67:0/3012752992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafa7c0b90 con 0x55fafaaee2f0 2024-09-10T23:07:50.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d8a800640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005730 tx=0x7f3d7c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffbf6c30e1d7c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.025+0000 7f3d6be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:07:50.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c035070 con 0x55fafaaee2f0 2024-09-10T23:07:50.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9519+0+0 (secure 0 0 0) 0x7f3d7c005020 con 0x55fafaaee2f0 2024-09-10T23:07:50.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c0054b0 con 0x55fafaaee2f0 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c058050 con 0x55fafaaee2f0 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d9405f740 10 client.15951 fetch_fsmap learned FSMap version 31 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.026+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fafab090b0 con 0x55fafaaee2f0 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d7c0054b0 con 0x55fafaaee2f0 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3d7c003d00 con 0x55fafaaee2f0 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 10 client.15951 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 10 client.15951 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 20 client.15951 populate_metadata read hostname 'smithi067' 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab0a400 0x55fafab0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 4 client.15951 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.027+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fafaa65bf0 con 0x55fafab0a400 2024-09-10T23:07:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.028+0000 7f3d89e00640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab0a400 0x55fafab0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.028+0000 7f3d89e00640 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab0a400 0x55fafab0c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22989c3f18120dff server_cookie=4c304616b9d31ac0 in_seq=0 out_seq=0 2024-09-10T23:07:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.028+0000 7f3d6be00640 10 client.15951 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.029+0000 7f3d6be00640 1 -- 172.21.15.67:0/3012752992 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fafaa65bf0 con 0x55fafab0a400 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.029+0000 7f3d6be00640 10 client.15951 handle_command_reply: tid=0 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.029+0000 7f3d9405f740 1 client.15951 shutdown 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 msgr2=0x55fafaafa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005730 tx=0x7f3d7c005900 comp rx=0 tx=0).stop 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 shutdown_connections 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab0a400 0x55fafab0c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaeec10 0x55fafaaf78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d6be00640 0 client.15951 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:07:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 --2- 172.21.15.67:0/3012752992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaee2f0 0x55fafaafa450 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 >> 172.21.15.67:0/3012752992 conn(0x55fafaaec120 msgr2=0x55fafaaf1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 shutdown_connections 2024-09-10T23:07:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 1 -- 172.21.15.67:0/3012752992 wait complete. 2024-09-10T23:07:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.030+0000 7f3d9405f740 20 client.15951 trim_cache size 0 max 16384 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 Processor -- start 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 -- start start 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 0x55fafaaf4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1c1380 con 0x55fafaaeec10 2024-09-10T23:07:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1bec50 con 0x55fafaafaba0 2024-09-10T23:07:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.058+0000 7f3d9405f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1bcac0 con 0x55fafaaf6580 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55386/0 (socket says 172.21.15.67:55386) 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 -- 172.21.15.67:0/4085698926 learned_addr learned my addr 172.21.15.67:0/4085698926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d89e00640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8b200640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 0x55fafaaf4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 -- 172.21.15.67:0/4085698926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 msgr2=0x55fafaaf4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 0x55fafaaf4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 -- 172.21.15.67:0/4085698926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 msgr2=0x55fafaaf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.059+0000 7f3d8a800640 1 -- 172.21.15.67:0/4085698926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafaaf81d0 con 0x55fafaaeec10 2024-09-10T23:07:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d8b200640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 0x55fafaaf4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:07:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d89e00640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d8a800640 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c03c5d0 tx=0x7f3d7c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e312cb7e423762 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d76a00640 1 -- 172.21.15.67:0/4085698926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c035070 con 0x55fafaaeec10 2024-09-10T23:07:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d76a00640 1 -- 172.21.15.67:0/4085698926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d7c061070 con 0x55fafaaeec10 2024-09-10T23:07:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.060+0000 7f3d76a00640 1 -- 172.21.15.67:0/4085698926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c040040 con 0x55fafaaeec10 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 -- 172.21.15.67:0/4085698926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 msgr2=0x55fafab069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c03c5d0 tx=0x7f3d7c005dd0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 -- 172.21.15.67:0/4085698926 shutdown_connections 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafaaf6580 0x55fafaaf4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafaaf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 --2- 172.21.15.67:0/4085698926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaeec10 0x55fafab069f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.061+0000 7f3d9405f740 1 -- 172.21.15.67:0/4085698926 >> 172.21.15.67:0/4085698926 conn(0x55fafaaec120 msgr2=0x55fafaaf96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.062+0000 7f3d9405f740 1 -- 172.21.15.67:0/4085698926 shutdown_connections 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.062+0000 7f3d9405f740 1 -- 172.21.15.67:0/4085698926 wait complete. 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.062+0000 7f3d9405f740 1 -- start start 2024-09-10T23:07:50.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.062+0000 7f3d9405f740 1 Processor -- start 2024-09-10T23:07:50.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 0x55fafab00450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 0x55fafab703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1bd110 con 0x55fafaaf6580 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1be580 con 0x55fafaafaba0 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1bd7e0 con 0x55fafab6ff60 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d9405f740 10 client.0 _ll_register_callbacks cb 0x7f3d7c0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 0x55fafab00450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.063+0000 7f3d89e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59704/0 (socket says 172.21.15.67:59704) 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 -- 172.21.15.67:0/946501071 learned_addr learned my addr 172.21.15.67:0/946501071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d8b200640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 0x55fafab703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 -- 172.21.15.67:0/946501071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 msgr2=0x55fafab703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 0x55fafab703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 -- 172.21.15.67:0/946501071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 msgr2=0x55fafab00450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 0x55fafab00450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.064+0000 7f3d89e00640 1 -- 172.21.15.67:0/946501071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafab70a80 con 0x55fafaafaba0 2024-09-10T23:07:50.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d8b200640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 0x55fafab703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:50.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d8a800640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 0x55fafab00450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d89e00640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d78020770 tx=0x7f3d78005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae0e8361bd8c98b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d6be00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780037f0 con 0x55fafaafaba0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c061070 con 0x55fafaafaba0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.065+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9519+0+0 (secure 0 0 0) 0x7f3d78003990 con 0x55fafaafaba0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.066+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d78003b90 con 0x55fafaafaba0 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.066+0000 7f3d9405f740 10 client.35675 fetch_fsmap learned FSMap version 31 2024-09-10T23:07:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.066+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fafab79000 con 0x55fafaafaba0 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.066+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d7c061070 con 0x55fafaafaba0 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.066+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3d7802cc80 con 0x55fafaafaba0 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 10 client.35675 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 10 client.35675 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 20 client.35675 populate_metadata read hostname 'smithi067' 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab79820 0x55fafab7bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 4 client.35675 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}] 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}) -- 0x55fafaa616a0 con 0x55fafab79820 2024-09-10T23:07:50.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.067+0000 7f3d8a800640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab79820 0x55fafab7bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.068+0000 7f3d8a800640 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab79820 0x55fafab7bc40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9f517036ba8c78 server_cookie=4918e93ae08deda2 in_seq=0 out_seq=0 2024-09-10T23:07:50.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.068+0000 7f3d6be00640 10 client.35675 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.192+0000 7f3d6be00640 1 -- 172.21.15.67:0/946501071 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fafaa616a0 con 0x55fafab79820 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.192+0000 7f3d6be00640 10 client.35675 handle_command_reply: tid=0 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.192+0000 7f3d9405f740 1 client.35675 shutdown 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 msgr2=0x55fafab009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d78020770 tx=0x7f3d78005f90 comp rx=0 tx=0).stop 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 shutdown_connections 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55fafab79820 0x55fafab7bc40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fafab6ff60 0x55fafab703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fafaafaba0 0x55fafab009f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d6be00640 0 client.35675 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 --2- 172.21.15.67:0/946501071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fafaaf6580 0x55fafab00450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 >> 172.21.15.67:0/946501071 conn(0x55fafaaec120 msgr2=0x55fafaaeaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 shutdown_connections 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 1 -- 172.21.15.67:0/946501071 wait complete. 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.193+0000 7f3d9405f740 20 client.35675 trim_cache size 0 max 16384 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:07:50.195 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 26, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "416b2ea": { 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 26, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 148.4, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:07:50.196 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:07:50.197 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 msgr2=0x7f3d700b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 0x7f3d700b7a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d780058c0 tx=0x7f3d780046c0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 msgr2=0x7f3d8c196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400eaa0 tx=0x7f3d8400ef60 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 shutdown_connections 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3d700b5580 0x7f3d700b7a40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d8c107e90 0x7f3d8c196ec0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d8c107510 0x7f3d8c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 --2- 172.21.15.67:0/1317003867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d8c101b00 0x7f3d8c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 >> 172.21.15.67:0/1317003867 conn(0x7f3d8c0fd5b0 msgr2=0x7f3d8c06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 shutdown_connections 2024-09-10T23:07:50.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:07:50.194+0000 7f3d90a00640 1 -- 172.21.15.67:0/1317003867 wait complete. 2024-09-10T23:07:51.377 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '416b2ea', quiesce took 4.9 seconds. Will release after: 23.5 2024-09-10T23:07:51.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:51 smithi145 ceph-mon[34397]: pgmap v660: 129 pgs: 129 active+clean; 4.1 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:07:51.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:51 smithi092 ceph-mon[34223]: pgmap v660: 129 pgs: 129 active+clean; 4.1 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:07:51.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:51 smithi067 ceph-mon[30939]: pgmap v660: 129 pgs: 129 active+clean; 4.1 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:07:53.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:53 smithi145 ceph-mon[34397]: pgmap v661: 129 pgs: 129 active+clean; 4.1 GiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:07:53.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:53 smithi092 ceph-mon[34223]: pgmap v661: 129 pgs: 129 active+clean; 4.1 GiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:07:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:53 smithi067 ceph-mon[30939]: pgmap v661: 129 pgs: 129 active+clean; 4.1 GiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:07:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:55 smithi145 ceph-mon[34397]: pgmap v662: 129 pgs: 129 active+clean; 4.1 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:55.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:55 smithi092 ceph-mon[34223]: pgmap v662: 129 pgs: 129 active+clean; 4.1 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:55.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:55 smithi067 ceph-mon[30939]: pgmap v662: 129 pgs: 129 active+clean; 4.1 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:07:57.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:57 smithi145 ceph-mon[34397]: pgmap v663: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:07:57.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:57 smithi092 ceph-mon[34223]: pgmap v663: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:07:57.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:57 smithi067 ceph-mon[30939]: pgmap v663: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:07:59.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:07:59 smithi145 ceph-mon[34397]: pgmap v664: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 8 op/s 2024-09-10T23:07:59.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:07:59 smithi092 ceph-mon[34223]: pgmap v664: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 8 op/s 2024-09-10T23:07:59.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:07:59 smithi067 ceph-mon[30939]: pgmap v664: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 8 op/s 2024-09-10T23:08:01.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:01 smithi145 ceph-mon[34397]: pgmap v665: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-10T23:08:01.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:01 smithi092 ceph-mon[34223]: pgmap v665: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-10T23:08:01.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:01 smithi067 ceph-mon[30939]: pgmap v665: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 10 op/s 2024-09-10T23:08:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:03 smithi145 ceph-mon[34397]: pgmap v666: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 21 op/s 2024-09-10T23:08:03.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:03 smithi092 ceph-mon[34223]: pgmap v666: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 21 op/s 2024-09-10T23:08:03.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:03 smithi067 ceph-mon[30939]: pgmap v666: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 21 op/s 2024-09-10T23:08:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:05 smithi145 ceph-mon[34397]: pgmap v667: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-09-10T23:08:05.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:05 smithi092 ceph-mon[34223]: pgmap v667: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-09-10T23:08:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:05 smithi067 ceph-mon[30939]: pgmap v667: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-09-10T23:08:07.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:07 smithi145 ceph-mon[34397]: pgmap v668: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 575 KiB/s wr, 24 op/s 2024-09-10T23:08:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:07 smithi092 ceph-mon[34223]: pgmap v668: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 575 KiB/s wr, 24 op/s 2024-09-10T23:08:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:07 smithi067 ceph-mon[30939]: pgmap v668: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 575 KiB/s wr, 24 op/s 2024-09-10T23:08:09.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:09 smithi067 ceph-mon[30939]: pgmap v669: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-10T23:08:09.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:09 smithi067 ceph-mon[30939]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T23:08:09.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:09 smithi145 ceph-mon[34397]: pgmap v669: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-10T23:08:09.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:09 smithi145 ceph-mon[34397]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T23:08:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:09 smithi092 ceph-mon[34223]: pgmap v669: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-10T23:08:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:09 smithi092 ceph-mon[34223]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T23:08:11.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:11 smithi067 ceph-mon[30939]: pgmap v671: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:08:11.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:11 smithi145 ceph-mon[34397]: pgmap v671: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:08:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:11 smithi092 ceph-mon[34223]: pgmap v671: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:08:13.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:13 smithi145 ceph-mon[34397]: pgmap v672: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:08:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:13 smithi092 ceph-mon[34223]: pgmap v672: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:08:13.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:13 smithi067 ceph-mon[30939]: pgmap v672: 129 pgs: 129 active+clean; 4.1 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:08:14.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:08:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:15.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:15 smithi145 ceph-mon[34397]: pgmap v673: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T23:08:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:15 smithi092 ceph-mon[34223]: pgmap v673: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T23:08:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:15 smithi067 ceph-mon[30939]: pgmap v673: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T23:08:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 -- 172.21.15.67:0/3523417270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f010a800 msgr2=0x7fc1f010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 --2- 172.21.15.67:0/3523417270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f010a800 0x7fc1f010ac40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009900 tx=0x7fc1d802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 -- 172.21.15.67:0/3523417270 shutdown_connections 2024-09-10T23:08:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 --2- 172.21.15.67:0/3523417270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 0x7fc1f0111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 --2- 172.21.15.67:0/3523417270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f010a800 0x7fc1f010ac40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 --2- 172.21.15.67:0/3523417270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f0104df0 0x7fc1f01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.370+0000 7fc1efe00640 1 -- 172.21.15.67:0/3523417270 >> 172.21.15.67:0/3523417270 conn(0x7fc1f0100840 msgr2=0x7fc1f0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.371+0000 7fc1efe00640 1 -- 172.21.15.67:0/3523417270 shutdown_connections 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.371+0000 7fc1efe00640 1 -- 172.21.15.67:0/3523417270 wait complete. 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.371+0000 7fc1efe00640 1 Processor -- start 2024-09-10T23:08:16.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 -- start start 2024-09-10T23:08:16.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:16.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f010a800 0x7fc1f019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 0x7fc1f01a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0114cf0 con 0x7fc1f0104df0 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0114b70 con 0x7fc1f010a800 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1efe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0114e70 con 0x7fc1f010b180 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1eea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1eea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39594/0 (socket says 172.21.15.67:39594) 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1eea00640 1 -- 172.21.15.67:0/556366659 learned_addr learned my addr 172.21.15.67:0/556366659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.372+0000 7fc1e6000640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f010a800 0x7fc1f019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1ef400640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 0x7fc1f01a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 -- 172.21.15.67:0/556366659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 msgr2=0x7fc1f01a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 0x7fc1f01a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 -- 172.21.15.67:0/556366659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f010a800 msgr2=0x7fc1f019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f010a800 0x7fc1f019b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 -- 172.21.15.67:0/556366659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f01a11b0 con 0x7fc1f0104df0 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.373+0000 7fc1eea00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d950 tx=0x7fc1e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:16.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.374+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0017070 con 0x7fc1f0104df0 2024-09-10T23:08:16.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.374+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1e0009c30 con 0x7fc1f0104df0 2024-09-10T23:08:16.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.374+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f01a14a0 con 0x7fc1f0104df0 2024-09-10T23:08:16.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.374+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000f620 con 0x7fc1f0104df0 2024-09-10T23:08:16.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.374+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f01a7b60 con 0x7fc1f0104df0 2024-09-10T23:08:16.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.375+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f01051d0 con 0x7fc1f0104df0 2024-09-10T23:08:16.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.380+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc1e0005030 con 0x7fc1f0104df0 2024-09-10T23:08:16.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.381+0000 7fc1e7e00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 0x7fc1d40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:16.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.381+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9607+0+0 (secure 0 0 0) 0x7fc1e00e5fe0 con 0x7fc1f0104df0 2024-09-10T23:08:16.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.381+0000 7fc1e6000640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 0x7fc1d40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.382+0000 7fc1e6000640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 0x7fc1d40b79c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80057c0 tx=0x7fc1d80058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.382+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc1e0009de0 con 0x7fc1f0104df0 2024-09-10T23:08:16.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.622+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f01a7eb0 con 0x7fc1f0104df0 2024-09-10T23:08:16.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.623+0000 7fc1e7e00640 1 -- 172.21.15.67:0/556366659 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fc1e00b3e20 con 0x7fc1f0104df0 2024-09-10T23:08:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:08:16.625 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:08:16.625 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 msgr2=0x7fc1d40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 0x7fc1d40b79c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80057c0 tx=0x7fc1d80058d0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 msgr2=0x7fc1f019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000d950 tx=0x7fc1e000de10 comp rx=0 tx=0).stop 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 shutdown_connections 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc1d40b5500 0x7fc1d40b79c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f010b180 0x7fc1f01a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1f010a800 0x7fc1f019b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 --2- 172.21.15.67:0/556366659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f0104df0 0x7fc1f019b100 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 >> 172.21.15.67:0/556366659 conn(0x7fc1f0100840 msgr2=0x7fc1f0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 shutdown_connections 2024-09-10T23:08:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:16.626+0000 7fc1efe00640 1 -- 172.21.15.67:0/556366659 wait complete. 2024-09-10T23:08:17.380 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 416b2ea --release --await' 2024-09-10T23:08:17.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 416b2ea --release --await 2024-09-10T23:08:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:17 smithi145 ceph-mon[34397]: pgmap v674: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:17.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/556366659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:17.761 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:17 smithi092 ceph-mon[34223]: pgmap v674: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:17.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/556366659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:17 smithi067 ceph-mon[30939]: pgmap v674: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/556366659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:18.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.408+0000 7f24d7a00640 1 -- 172.21.15.67:0/2605652089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075f40 msgr2=0x7f24d010ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.408+0000 7f24d7a00640 1 --2- 172.21.15.67:0/2605652089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075f40 0x7f24d010ee10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009900 tx=0x7f24bc02a410 comp rx=0 tx=0).stop 2024-09-10T23:08:18.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 -- 172.21.15.67:0/2605652089 shutdown_connections 2024-09-10T23:08:18.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 --2- 172.21.15.67:0/2605652089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d010f3e0 0x7f24d01117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 --2- 172.21.15.67:0/2605652089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075f40 0x7f24d010ee10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 --2- 172.21.15.67:0/2605652089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d0075540 0x7f24d0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 -- 172.21.15.67:0/2605652089 >> 172.21.15.67:0/2605652089 conn(0x7f24d00fe830 msgr2=0x7f24d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 -- 172.21.15.67:0/2605652089 shutdown_connections 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.409+0000 7f24d7a00640 1 -- 172.21.15.67:0/2605652089 wait complete. 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 Processor -- start 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 -- start start 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d0075f40 0x7f24d019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d010f3e0 0x7f24d01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0114e40 con 0x7f24d0075540 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0114cc0 con 0x7f24d0075f40 2024-09-10T23:08:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d7a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0114fc0 con 0x7f24d010f3e0 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.410+0000 7f24d5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39618/0 (socket says 172.21.15.67:39618) 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 -- 172.21.15.67:0/3706118153 learned_addr learned my addr 172.21.15.67:0/3706118153 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d010f3e0 0x7f24d01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24cfe00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d0075f40 0x7f24d019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 -- 172.21.15.67:0/3706118153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d010f3e0 msgr2=0x7f24d01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d010f3e0 0x7f24d01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 -- 172.21.15.67:0/3706118153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d0075f40 msgr2=0x7f24d019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d0075f40 0x7f24d019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 -- 172.21.15.67:0/3706118153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d01a14e0 con 0x7f24d0075540 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.411+0000 7f24d5000640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.412+0000 7f24cd600640 1 -- 172.21.15.67:0/3706118153 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0017070 con 0x7f24d0075540 2024-09-10T23:08:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.412+0000 7f24cd600640 1 -- 172.21.15.67:0/3706118153 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c0009c30 con 0x7f24d0075540 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.412+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d01a17d0 con 0x7f24d0075540 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.412+0000 7f24cd600640 1 -- 172.21.15.67:0/3706118153 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000f620 con 0x7f24d0075540 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.412+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d01a7e90 con 0x7f24d0075540 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 Processor -- start 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 -- start start 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24cd600640 1 -- 172.21.15.67:0/3706118153 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f24c0010460 con 0x7f24d0075540 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e5910 0x5601a84f1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a84f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5601a845c290 con 0x5601a84e4ff0 2024-09-10T23:08:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601a844b6c0 con 0x5601a84e5910 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.415+0000 7f24da673740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5601a844f4d0 con 0x5601a84f18a0 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39622/0 (socket says 172.21.15.67:39622) 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 -- 172.21.15.67:0/3303621091 learned_addr learned my addr 172.21.15.67:0/3303621091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 -- 172.21.15.67:0/3303621091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 msgr2=0x5601a84f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a84f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 -- 172.21.15.67:0/3303621091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e5910 msgr2=0x5601a84f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e5910 0x5601a84f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 -- 172.21.15.67:0/3303621091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a83d8460 con 0x5601a84e4ff0 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.416+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f24c80103a0 tx=0x7f24c8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78605dec9becef95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24c6000640 1 -- 172.21.15.67:0/3303621091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8016070 con 0x5601a84e4ff0 2024-09-10T23:08:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24c6000640 1 -- 172.21.15.67:0/3303621091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c8004d20 con 0x5601a84e4ff0 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24c6000640 1 -- 172.21.15.67:0/3303621091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8005370 con 0x5601a84e4ff0 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24cd600640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 0x7f24b40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24cd600640 1 -- 172.21.15.67:0/3706118153 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9607+0+0 (secure 0 0 0) 0x7f24c00e60c0 con 0x7f24d0075540 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24d5000640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 0x7f24b40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24da673740 1 -- 172.21.15.67:0/3303621091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 msgr2=0x5601a84e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.417+0000 7f24da673740 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f24c80103a0 tx=0x7f24c8010860 comp rx=0 tx=0).stop 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 -- 172.21.15.67:0/3303621091 shutdown_connections 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a84f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e5910 0x5601a84f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 --2- 172.21.15.67:0/3303621091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e4ff0 0x5601a84e53d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 -- 172.21.15.67:0/3303621091 >> 172.21.15.67:0/3303621091 conn(0x5601a84e2e20 msgr2=0x5601a84e3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 -- 172.21.15.67:0/3303621091 shutdown_connections 2024-09-10T23:08:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 -- 172.21.15.67:0/3303621091 wait complete. 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24d5000640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 0x7f24b40b7910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5601a84e8170 tx=0x7f24c00b5050 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 -- start start 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.418+0000 7f24da673740 1 Processor -- start 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e4ff0 0x5601a84ee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84f18a0 0x5601a84ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5601a82acbb0 con 0x5601a84e5910 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601a83e66c0 con 0x5601a84f18a0 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5601a84f4550 con 0x5601a84e4ff0 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24da673740 10 client.0 _ll_register_callbacks cb 0x5601a8401490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39638/0 (socket says 172.21.15.67:39638) 2024-09-10T23:08:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24cfe00640 1 -- 172.21.15.67:0/251551808 learned_addr learned my addr 172.21.15.67:0/251551808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.419+0000 7f24d5a00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e4ff0 0x5601a84ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24d5a00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84f18a0 0x5601a84ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 -- 172.21.15.67:0/251551808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e4ff0 msgr2=0x5601a84ee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e4ff0 0x5601a84ee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 -- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84f18a0 msgr2=0x5601a84ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84f18a0 0x5601a84ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 -- 172.21.15.67:0/251551808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a81a9b00 con 0x5601a84e5910 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24cfe00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005ab0 tx=0x7f24bc031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83485d1fface506 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24baa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.420+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc03f070 con 0x5601a84e5910 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9607+0+0 (secure 0 0 0) 0x7f24bc004250 con 0x5601a84e5910 2024-09-10T23:08:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8004d20 con 0x5601a84e5910 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f24bc035070 con 0x5601a84e5910 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 10 client.15975 fetch_fsmap learned FSMap version 31 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601a8220960 con 0x5601a84e5910 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c8004d20 con 0x5601a84e5910 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f24bc0048c0 con 0x5601a84e5910 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 10 client.15975 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 20 client.15975 populate_metadata read hostname 'smithi067' 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a8501430 0x5601a8503850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 4 client.15975 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.421+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a845e1e0 con 0x5601a8501430 2024-09-10T23:08:18.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.422+0000 7f24d5a00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a8501430 0x5601a8503850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.423+0000 7f24d5a00640 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a8501430 0x5601a8503850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d54186642cd49ee server_cookie=4a943f576b48be6 in_seq=0 out_seq=0 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.423+0000 7f24baa00640 10 client.15975 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.423+0000 7f24baa00640 1 -- 172.21.15.67:0/251551808 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601a845e1e0 con 0x5601a8501430 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.423+0000 7f24baa00640 10 client.15975 handle_command_reply: tid=0 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.423+0000 7f24da673740 1 client.15975 shutdown 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 msgr2=0x5601a84eed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005ab0 tx=0x7f24bc031630 comp rx=0 tx=0).stop 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 shutdown_connections 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a8501430 0x5601a8503850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84f18a0 0x5601a84ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84eed60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 --2- 172.21.15.67:0/251551808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e4ff0 0x5601a84ee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 >> 172.21.15.67:0/251551808 conn(0x5601a84e2e20 msgr2=0x5601a84e89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24baa00640 0 client.15975 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:08:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 shutdown_connections 2024-09-10T23:08:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 1 -- 172.21.15.67:0/251551808 wait complete. 2024-09-10T23:08:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.424+0000 7f24da673740 20 client.15975 trim_cache size 0 max 16384 2024-09-10T23:08:18.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 Processor -- start 2024-09-10T23:08:18.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 -- start start 2024-09-10T23:08:18.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e5910 0x5601a84ebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84f18a0 0x5601a85090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b4ae0 con 0x5601a84f18a0 2024-09-10T23:08:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b23b0 con 0x5601a84e4ff0 2024-09-10T23:08:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24da673740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b0220 con 0x5601a84e5910 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84f18a0 0x5601a85090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e5910 0x5601a84ebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24d5a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:37920/0 (socket says 172.21.15.67:37920) 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.443+0000 7f24d5a00640 1 -- 172.21.15.67:0/3004224836 learned_addr learned my addr 172.21.15.67:0/3004224836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 -- 172.21.15.67:0/3004224836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e5910 msgr2=0x5601a84ebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e5910 0x5601a84ebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 -- 172.21.15.67:0/3004224836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84f18a0 msgr2=0x5601a85090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84f18a0 0x5601a85090e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 -- 172.21.15.67:0/3004224836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a8503eb0 con 0x5601a84e4ff0 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.444+0000 7f24d5a00640 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005fd0 tx=0x7f24c8010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8327860726d1d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24c5600640 1 -- 172.21.15.67:0/3004224836 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8016040 con 0x5601a84e4ff0 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24c5600640 1 -- 172.21.15.67:0/3004224836 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c8005020 con 0x5601a84e4ff0 2024-09-10T23:08:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24c5600640 1 -- 172.21.15.67:0/3004224836 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c802a6c0 con 0x5601a84e4ff0 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 -- 172.21.15.67:0/3004224836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 msgr2=0x5601a84eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005fd0 tx=0x7f24c8010fd0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 -- 172.21.15.67:0/3004224836 shutdown_connections 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84f18a0 0x5601a85090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84e5910 0x5601a84ebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 --2- 172.21.15.67:0/3004224836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84eba30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 -- 172.21.15.67:0/3004224836 >> 172.21.15.67:0/3004224836 conn(0x5601a84e2e20 msgr2=0x5601a84e89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 -- 172.21.15.67:0/3004224836 shutdown_connections 2024-09-10T23:08:18.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.445+0000 7f24da673740 1 -- 172.21.15.67:0/3004224836 wait complete. 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 -- start start 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 Processor -- start 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a856e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b0870 con 0x5601a84e5910 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b1ce0 con 0x5601a84e4ff0 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.446+0000 7f24da673740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b0f40 con 0x5601a84f18a0 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24da673740 10 client.0 _ll_register_callbacks cb 0x7f24bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39664/0 (socket says 172.21.15.67:39664) 2024-09-10T23:08:18.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 -- 172.21.15.67:0/1901134836 learned_addr learned my addr 172.21.15.67:0/1901134836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24d5a00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24d5a00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a856e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 -- 172.21.15.67:0/1901134836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 msgr2=0x5601a856e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a856e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 -- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 msgr2=0x5601a84f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24cfe00640 1 -- 172.21.15.67:0/1901134836 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a84e9540 con 0x5601a84e5910 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.447+0000 7f24d5a00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a856e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24cfe00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f24bc02a930 tx=0x7f24bc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b98502eab4bbb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24baa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc03f070 con 0x5601a84e5910 2024-09-10T23:08:18.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9607+0+0 (secure 0 0 0) 0x7f24bc0048c0 con 0x5601a84e5910 2024-09-10T23:08:18.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8005020 con 0x5601a84e5910 2024-09-10T23:08:18.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f24bc063050 con 0x5601a84e5910 2024-09-10T23:08:18.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24da673740 10 client.15981 fetch_fsmap learned FSMap version 31 2024-09-10T23:08:18.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601a84e49d0 con 0x5601a84e5910 2024-09-10T23:08:18.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.448+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c8005020 con 0x5601a84e5910 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f24bc005200 con 0x5601a84e5910 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 10 client.15981 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 10 client.15981 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 20 client.15981 populate_metadata read hostname 'smithi067' 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a856ff30 0x5601a8572350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 4 client.15981 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "416b2ea", "release": true, "await": true}] 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "416b2ea", "release": true, "await": true}) -- 0x5601a8459c90 con 0x5601a856ff30 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.449+0000 7f24d5a00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a856ff30 0x5601a8572350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.450+0000 7f24d5a00640 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a856ff30 0x5601a8572350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8198d624ad03e8 server_cookie=c58386c2e5fdd6cf in_seq=0 out_seq=0 2024-09-10T23:08:18.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.450+0000 7f24baa00640 10 client.15981 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:08:18.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24baa00640 1 -- 172.21.15.67:0/1901134836 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5601a8459c90 con 0x5601a856ff30 2024-09-10T23:08:18.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24baa00640 10 client.15981 handle_command_reply: tid=0 2024-09-10T23:08:18.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 client.15981 shutdown 2024-09-10T23:08:18.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 msgr2=0x5601a84f7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f24bc02a930 tx=0x7f24bc009520 comp rx=0 tx=0).stop 2024-09-10T23:08:18.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 shutdown_connections 2024-09-10T23:08:18.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601a856ff30 0x5601a8572350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601a84f18a0 0x5601a856e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601a84e5910 0x5601a84f7140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 --2- 172.21.15.67:0/1901134836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601a84e4ff0 0x5601a84f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 >> 172.21.15.67:0/1901134836 conn(0x5601a84e2e20 msgr2=0x5601a84ed0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24baa00640 0 client.15981 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 shutdown_connections 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.472+0000 7f24da673740 1 -- 172.21.15.67:0/1901134836 wait complete. 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.473+0000 7f24da673740 20 client.15981 trim_cache size 0 max 16384 2024-09-10T23:08:18.475 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 28, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "416b2ea": { 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 28, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:08:18.476 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 148.4, 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:08:18.477 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.473+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 msgr2=0x7f24b40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.473+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 0x7f24b40b7910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5601a84e8170 tx=0x7f24c00b5050 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.473+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 msgr2=0x7f24d019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.473+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 shutdown_connections 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f24b40b5450 0x7f24b40b7910 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5601a84e8170 tx=0x7f24c00b5050 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24d010f3e0 0x7f24d01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d0075f40 0x7f24d019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 --2- 172.21.15.67:0/3706118153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24d0075540 0x7f24d019b430 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).stop 2024-09-10T23:08:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 >> 172.21.15.67:0/3706118153 conn(0x7f24d00fe830 msgr2=0x7f24d00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 shutdown_connections 2024-09-10T23:08:18.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:08:18.474+0000 7f24d7a00640 1 -- 172.21.15.67:0/3706118153 wait complete. 2024-09-10T23:08:19.358 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '416b2ea', total seconds elapsed: 32.9 2024-09-10T23:08:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:19 smithi092 ceph-mon[34223]: pgmap v675: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:19 smithi092 ceph-mon[34223]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T23:08:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:19 smithi067 ceph-mon[30939]: pgmap v675: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:19 smithi067 ceph-mon[30939]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T23:08:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:19 smithi145 ceph-mon[34397]: pgmap v675: 129 pgs: 129 active+clean; 4.1 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:08:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:19 smithi145 ceph-mon[34397]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T23:08:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:21 smithi092 ceph-mon[34223]: pgmap v677: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 KiB/s wr, 6 op/s 2024-09-10T23:08:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:21 smithi067 ceph-mon[30939]: pgmap v677: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 KiB/s wr, 6 op/s 2024-09-10T23:08:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:21 smithi145 ceph-mon[34397]: pgmap v677: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 KiB/s wr, 6 op/s 2024-09-10T23:08:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:23 smithi092 ceph-mon[34223]: pgmap v678: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1006 KiB/s wr, 6 op/s 2024-09-10T23:08:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:23 smithi067 ceph-mon[30939]: pgmap v678: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1006 KiB/s wr, 6 op/s 2024-09-10T23:08:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:23 smithi145 ceph-mon[34397]: pgmap v678: 129 pgs: 129 active+clean; 4.1 GiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1006 KiB/s wr, 6 op/s 2024-09-10T23:08:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:25 smithi092 ceph-mon[34223]: pgmap v679: 129 pgs: 129 active+clean; 4.2 GiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.8 MiB/s wr, 8 op/s 2024-09-10T23:08:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:25 smithi067 ceph-mon[30939]: pgmap v679: 129 pgs: 129 active+clean; 4.2 GiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.8 MiB/s wr, 8 op/s 2024-09-10T23:08:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:25 smithi145 ceph-mon[34397]: pgmap v679: 129 pgs: 129 active+clean; 4.2 GiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.8 MiB/s wr, 8 op/s 2024-09-10T23:08:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:27 smithi092 ceph-mon[34223]: pgmap v680: 129 pgs: 129 active+clean; 4.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.4 MiB/s wr, 9 op/s 2024-09-10T23:08:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:27 smithi067 ceph-mon[30939]: pgmap v680: 129 pgs: 129 active+clean; 4.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.4 MiB/s wr, 9 op/s 2024-09-10T23:08:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:27 smithi145 ceph-mon[34397]: pgmap v680: 129 pgs: 129 active+clean; 4.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.4 MiB/s wr, 9 op/s 2024-09-10T23:08:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:29 smithi092 ceph-mon[34223]: pgmap v681: 129 pgs: 129 active+clean; 4.2 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.8 MiB/s wr, 9 op/s 2024-09-10T23:08:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:29 smithi067 ceph-mon[30939]: pgmap v681: 129 pgs: 129 active+clean; 4.2 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.8 MiB/s wr, 9 op/s 2024-09-10T23:08:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:29 smithi145 ceph-mon[34397]: pgmap v681: 129 pgs: 129 active+clean; 4.2 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.8 MiB/s wr, 9 op/s 2024-09-10T23:08:31.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:31 smithi092 ceph-mon[34223]: pgmap v682: 129 pgs: 129 active+clean; 4.3 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:08:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:31 smithi067 ceph-mon[30939]: pgmap v682: 129 pgs: 129 active+clean; 4.3 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:08:31.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:31 smithi145 ceph-mon[34397]: pgmap v682: 129 pgs: 129 active+clean; 4.3 GiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:08:33.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:33 smithi092 ceph-mon[34223]: pgmap v683: 129 pgs: 129 active+clean; 4.3 GiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:33.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:33 smithi067 ceph-mon[30939]: pgmap v683: 129 pgs: 129 active+clean; 4.3 GiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:33 smithi145 ceph-mon[34397]: pgmap v683: 129 pgs: 129 active+clean; 4.3 GiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:35.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:35 smithi092 ceph-mon[34223]: pgmap v684: 129 pgs: 129 active+clean; 4.3 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:08:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:35 smithi067 ceph-mon[30939]: pgmap v684: 129 pgs: 129 active+clean; 4.3 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:08:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:35 smithi145 ceph-mon[34397]: pgmap v684: 129 pgs: 129 active+clean; 4.3 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:08:37.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:37 smithi092 ceph-mon[34223]: pgmap v685: 129 pgs: 129 active+clean; 4.3 GiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:08:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:37 smithi067 ceph-mon[30939]: pgmap v685: 129 pgs: 129 active+clean; 4.3 GiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:08:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:37 smithi145 ceph-mon[34397]: pgmap v685: 129 pgs: 129 active+clean; 4.3 GiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:08:39.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:39 smithi092 ceph-mon[34223]: pgmap v686: 129 pgs: 129 active+clean; 4.3 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:08:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:39 smithi067 ceph-mon[30939]: pgmap v686: 129 pgs: 129 active+clean; 4.3 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:08:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:39 smithi145 ceph-mon[34397]: pgmap v686: 129 pgs: 129 active+clean; 4.3 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:08:41.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:41 smithi092 ceph-mon[34223]: pgmap v687: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:08:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:41 smithi067 ceph-mon[30939]: pgmap v687: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:08:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:41 smithi145 ceph-mon[34397]: pgmap v687: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:08:43.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:43 smithi092 ceph-mon[34223]: pgmap v688: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:08:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:43 smithi067 ceph-mon[30939]: pgmap v688: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:08:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:43 smithi145 ceph-mon[34397]: pgmap v688: 129 pgs: 129 active+clean; 4.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:08:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:45 smithi067 ceph-mon[30939]: pgmap v689: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:45 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:45 smithi145 ceph-mon[34397]: pgmap v689: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:45 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:45 smithi092 ceph-mon[34223]: pgmap v689: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:45 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:46 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:46 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:46 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:47 smithi092 ceph-mon[34223]: pgmap v690: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:47 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:47 smithi067 ceph-mon[30939]: pgmap v690: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:47 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:47 smithi145 ceph-mon[34397]: pgmap v690: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:08:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:47 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:49 smithi092 ceph-mon[34223]: pgmap v691: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:08:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:49 smithi067 ceph-mon[30939]: pgmap v691: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:08:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:49 smithi145 ceph-mon[34397]: pgmap v691: 129 pgs: 129 active+clean; 4.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:08:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:51 smithi092 ceph-mon[34223]: pgmap v692: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:08:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:51 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:51 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:51 smithi067 ceph-mon[30939]: pgmap v692: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:08:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:51 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:51 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:51 smithi145 ceph-mon[34397]: pgmap v692: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:08:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:51 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:51 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:53.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:53.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:53.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:52 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:53.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:53.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:53.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:52 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:53.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:53.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:53.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:52 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:08:54.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:53 smithi092 ceph-mon[34223]: pgmap v693: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:54.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:53 smithi067 ceph-mon[30939]: pgmap v693: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:54.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:53 smithi145 ceph-mon[34397]: pgmap v693: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:08:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:55 smithi092 ceph-mon[34223]: pgmap v694: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:08:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:55 smithi067 ceph-mon[30939]: pgmap v694: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:08:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:55 smithi145 ceph-mon[34397]: pgmap v694: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:08:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:57 smithi092 ceph-mon[34223]: pgmap v695: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:08:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:57 smithi067 ceph-mon[30939]: pgmap v695: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:08:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:57 smithi145 ceph-mon[34397]: pgmap v695: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:09:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:08:59 smithi092 ceph-mon[34223]: pgmap v696: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:09:00.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:08:59 smithi067 ceph-mon[30939]: pgmap v696: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:09:00.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:08:59 smithi145 ceph-mon[34397]: pgmap v696: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:09:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:01 smithi092 ceph-mon[34223]: pgmap v697: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:01 smithi067 ceph-mon[30939]: pgmap v697: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:01 smithi145 ceph-mon[34397]: pgmap v697: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:04.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:03 smithi092 ceph-mon[34223]: pgmap v698: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:09:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:03 smithi067 ceph-mon[30939]: pgmap v698: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:09:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:03 smithi145 ceph-mon[34397]: pgmap v698: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:09:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:05 smithi092 ceph-mon[34223]: pgmap v699: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-09-10T23:09:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:05 smithi067 ceph-mon[30939]: pgmap v699: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-09-10T23:09:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:05 smithi145 ceph-mon[34397]: pgmap v699: 129 pgs: 129 active+clean; 4.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 28 op/s 2024-09-10T23:09:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:07 smithi092 ceph-mon[34223]: pgmap v700: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:09:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:07 smithi067 ceph-mon[30939]: pgmap v700: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:09:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:07 smithi145 ceph-mon[34397]: pgmap v700: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:09:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:08 smithi092 ceph-mon[34223]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T23:09:09.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:08 smithi145 ceph-mon[34397]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T23:09:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:08 smithi067 ceph-mon[30939]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T23:09:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:10 smithi092 ceph-mon[34223]: pgmap v702: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:09:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:10 smithi067 ceph-mon[30939]: pgmap v702: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:09:10.299 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:10 smithi145 ceph-mon[34397]: pgmap v702: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-09-10T23:09:12.102 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.4 2024-09-10T23:09:12.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:09:12.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:12 smithi092 ceph-mon[34223]: pgmap v703: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:09:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:12 smithi067 ceph-mon[30939]: pgmap v703: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:09:12.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:12 smithi145 ceph-mon[34397]: pgmap v703: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:09:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 -- 172.21.15.67:0/2666666668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8870075de0 msgr2=0x7f887010ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 --2- 172.21.15.67:0/2666666668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8870075de0 0x7f887010ece0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8858009900 tx=0x7f885802a410 comp rx=0 tx=0).stop 2024-09-10T23:09:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 -- 172.21.15.67:0/2666666668 shutdown_connections 2024-09-10T23:09:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 --2- 172.21.15.67:0/2666666668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887010f220 0x7f8870111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 --2- 172.21.15.67:0/2666666668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8870075de0 0x7f887010ece0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 --2- 172.21.15.67:0/2666666668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f88700758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 -- 172.21.15.67:0/2666666668 >> 172.21.15.67:0/2666666668 conn(0x7f88700fe660 msgr2=0x7f8870100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 -- 172.21.15.67:0/2666666668 shutdown_connections 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.146+0000 7f886f200640 1 -- 172.21.15.67:0/2666666668 wait complete. 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.147+0000 7f886f200640 1 Processor -- start 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.147+0000 7f886f200640 1 -- start start 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f887019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 0x7f887019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8870114d00 con 0x7f887010f220 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8870114b80 con 0x7f8870075de0 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8870114e80 con 0x7f88700754c0 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46196/0 (socket says 172.21.15.67:46196) 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f8867e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 0x7f887019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886de00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f887019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.148+0000 7f886e800640 1 -- 172.21.15.67:0/3375110819 learned_addr learned my addr 172.21.15.67:0/3375110819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 -- 172.21.15.67:0/3375110819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 msgr2=0x7f887019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f887019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 -- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 msgr2=0x7f88701a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 -- 172.21.15.67:0/3375110819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88701a1320 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f886de00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f887019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f886e800640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8867e00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 0x7f887019b7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f885802a930 tx=0x7f8858003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885803f070 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f885803baf0 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858035030 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.149+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88701a15b0 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.150+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701a7cd0 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.151+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88700758a0 con 0x7f8870075de0 2024-09-10T23:09:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.154+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f885804c070 con 0x7f8870075de0 2024-09-10T23:09:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.156+0000 7f8866a00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 0x7f88540b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.156+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9607+0+0 (secure 0 0 0) 0x7f8858107070 con 0x7f8870075de0 2024-09-10T23:09:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.156+0000 7f886de00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 0x7f88540b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.157+0000 7f886de00640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 0x7f88540b7a40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f886000bfd0 tx=0x7f886000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.157+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f88580cd100 con 0x7f8870075de0 2024-09-10T23:09:13.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.396+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88701a7fc0 con 0x7f8870075de0 2024-09-10T23:09:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.397+0000 7f8866a00640 1 -- 172.21.15.67:0/3375110819 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f88580d1b30 con 0x7f8870075de0 2024-09-10T23:09:13.399 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:09:13.399 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:09:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:09:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 msgr2=0x7f88540b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 0x7f88540b7a40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f886000bfd0 tx=0x7f886000b390 comp rx=0 tx=0).stop 2024-09-10T23:09:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 msgr2=0x7f887019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 0x7f887019b7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f885802a930 tx=0x7f8858003b90 comp rx=0 tx=0).stop 2024-09-10T23:09:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 shutdown_connections 2024-09-10T23:09:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f88540b5580 0x7f88540b7a40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887010f220 0x7f88701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870075de0 0x7f887019b7b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 --2- 172.21.15.67:0/3375110819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88700754c0 0x7f887019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 >> 172.21.15.67:0/3375110819 conn(0x7f88700fe660 msgr2=0x7f88701113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.400+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 shutdown_connections 2024-09-10T23:09:13.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:13.401+0000 7f886f200640 1 -- 172.21.15.67:0/3375110819 wait complete. 2024-09-10T23:09:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:14 smithi092 ceph-mon[34223]: pgmap v704: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:09:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:14 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3375110819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:14 smithi067 ceph-mon[30939]: pgmap v704: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:09:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:14 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3375110819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:14 smithi145 ceph-mon[34397]: pgmap v704: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:09:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:14 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3375110819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:14.575 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0' 2024-09-10T23:09:14.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0 2024-09-10T23:09:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 -- 172.21.15.67:0/938045516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108f90 msgr2=0x7f9a4010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 --2- 172.21.15.67:0/938045516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108f90 0x7f9a4010f790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009ca0 tx=0x7f9a3802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 -- 172.21.15.67:0/938045516 shutdown_connections 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 --2- 172.21.15.67:0/938045516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108f90 0x7f9a4010f790 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 --2- 172.21.15.67:0/938045516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40108610 0x7f9a40108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 --2- 172.21.15.67:0/938045516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40102c00 0x7f9a40102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.780+0000 7f9a45e00640 1 -- 172.21.15.67:0/938045516 >> 172.21.15.67:0/938045516 conn(0x7f9a400fe6d0 msgr2=0x7f9a40100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.781+0000 7f9a45e00640 1 -- 172.21.15.67:0/938045516 shutdown_connections 2024-09-10T23:09:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.781+0000 7f9a45e00640 1 -- 172.21.15.67:0/938045516 wait complete. 2024-09-10T23:09:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.782+0000 7f9a45e00640 1 Processor -- start 2024-09-10T23:09:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.782+0000 7f9a45e00640 1 -- start start 2024-09-10T23:09:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.782+0000 7f9a45e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102c00 0x7f9a4019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.782+0000 7f9a45e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a45e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40108f90 0x7f9a401a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a45e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112b50 con 0x7f9a40108610 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a45e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401129d0 con 0x7f9a40102c00 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a45e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112cd0 con 0x7f9a40108f90 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46216/0 (socket says 172.21.15.67:46216) 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 -- 172.21.15.67:0/3425624844 learned_addr learned my addr 172.21.15.67:0/3425624844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a45400640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40108f90 0x7f9a401a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102c00 0x7f9a4019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 -- 172.21.15.67:0/3425624844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40108f90 msgr2=0x7f9a401a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40108f90 0x7f9a401a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 -- 172.21.15.67:0/3425624844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102c00 msgr2=0x7f9a4019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102c00 0x7f9a4019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.783+0000 7f9a3fe00640 1 -- 172.21.15.67:0/3425624844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401a1190 con 0x7f9a40108610 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00ea60 tx=0x7f9a2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a3d600640 1 -- 172.21.15.67:0/3425624844 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00cde0 con 0x7f9a40108610 2024-09-10T23:09:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a3d600640 1 -- 172.21.15.67:0/3425624844 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a2c0035f0 con 0x7f9a40108610 2024-09-10T23:09:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a3d600640 1 -- 172.21.15.67:0/3425624844 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c010690 con 0x7f9a40108610 2024-09-10T23:09:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401a1480 con 0x7f9a40108610 2024-09-10T23:09:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.784+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401a7b40 con 0x7f9a40108610 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.788+0000 7f9a49dc3740 1 Processor -- start 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.788+0000 7f9a49dc3740 1 -- start start 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557db1f06ca0 con 0x557db1f91c10 2024-09-10T23:09:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557db1ef60d0 con 0x557db1f9dba0 2024-09-10T23:09:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557db1ef9ee0 con 0x557db1f912f0 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46226/0 (socket says 172.21.15.67:46226) 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a44a00640 1 -- 172.21.15.67:0/282219034 learned_addr learned my addr 172.21.15.67:0/282219034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.789+0000 7f9a45400640 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 -- 172.21.15.67:0/282219034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 msgr2=0x557db1f916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 -- 172.21.15.67:0/282219034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 msgr2=0x557db1f9ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 -- 172.21.15.67:0/282219034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db1ea8400 con 0x557db1f91c10 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a3d600640 1 -- 172.21.15.67:0/3425624844 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9a2c004000 con 0x7f9a40108610 2024-09-10T23:09:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.790+0000 7f9a44a00640 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009c90 tx=0x7f9a340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b101a72507d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.791+0000 7f9a31c00640 1 -- 172.21.15.67:0/282219034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34012450 con 0x557db1f91c10 2024-09-10T23:09:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.791+0000 7f9a31c00640 1 -- 172.21.15.67:0/282219034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a34012a80 con 0x557db1f91c10 2024-09-10T23:09:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.791+0000 7f9a31c00640 1 -- 172.21.15.67:0/282219034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400fb20 con 0x557db1f91c10 2024-09-10T23:09:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 -- 172.21.15.67:0/282219034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 msgr2=0x557db1f9d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009c90 tx=0x7f9a340117b0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 -- 172.21.15.67:0/282219034 shutdown_connections 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9d660 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/282219034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 -- 172.21.15.67:0/282219034 >> 172.21.15.67:0/282219034 conn(0x557db1f8f120 msgr2=0x557db1f902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 -- 172.21.15.67:0/282219034 shutdown_connections 2024-09-10T23:09:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.792+0000 7f9a49dc3740 1 -- 172.21.15.67:0/282219034 wait complete. 2024-09-10T23:09:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.793+0000 7f9a3d600640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 0x7f9a280b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.793+0000 7f9a49dc3740 1 -- start start 2024-09-10T23:09:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.793+0000 7f9a3d600640 1 -- 172.21.15.67:0/3425624844 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9607+0+0 (secure 0 0 0) 0x7f9a2c0e6710 con 0x7f9a40108610 2024-09-10T23:09:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.793+0000 7f9a49dc3740 1 Processor -- start 2024-09-10T23:09:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.793+0000 7f9a45400640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 0x7f9a280b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557db1d577a0 con 0x557db1f91c10 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557db1e9ba10 con 0x557db1f9dba0 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557db1fa0820 con 0x557db1f912f0 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a49dc3740 10 client.0 _ll_register_callbacks cb 0x557db1eabbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.794+0000 7f9a45400640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 0x7f9a280b7970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006010 tx=0x7f9a38004a80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:53144/0 (socket says 172.21.15.67:53144) 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 -- 172.21.15.67:0/772097070 learned_addr learned my addr 172.21.15.67:0/772097070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a45400640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 -- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 msgr2=0x557db1f9b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 -- 172.21.15.67:0/772097070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 msgr2=0x557db1f9a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a44a00640 1 -- 172.21.15.67:0/772097070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db1c63b90 con 0x557db1f912f0 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.795+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.796+0000 7f9a44a00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009860 tx=0x7f9a34005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af6ab898515226b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.796+0000 7f9a26000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34029030 con 0x557db1f912f0 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9607+0+0 (secure 0 0 0) 0x7f9a34020040 con 0x557db1f912f0 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a34012a80 con 0x557db1f912f0 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9a34033050 con 0x557db1f912f0 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a49dc3740 10 client.25714 fetch_fsmap learned FSMap version 31 2024-09-10T23:09:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.797+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557db1fb4a70 con 0x557db1f912f0 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.798+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a34012a80 con 0x557db1f912f0 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.798+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9a34004300 con 0x557db1f912f0 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.798+0000 7f9a49dc3740 10 client.25714 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.798+0000 7f9a49dc3740 10 client.25714 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.799+0000 7f9a49dc3740 20 client.25714 populate_metadata read hostname 'smithi067' 2024-09-10T23:09:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.799+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db1fb5ed0 0x557db1fb82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.799+0000 7f9a49dc3740 4 client.25714 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.799+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557db1f08bf0 con 0x557db1fb5ed0 2024-09-10T23:09:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.799+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db1fb5ed0 0x557db1fb82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.800+0000 7f9a26000640 10 client.25714 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.800+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db1fb5ed0 0x557db1fb82c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97045f8ce42b2c35 server_cookie=c58d24e7d4b27c8b in_seq=0 out_seq=0 2024-09-10T23:09:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a26000640 1 -- 172.21.15.67:0/772097070 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557db1f08bf0 con 0x557db1fb5ed0 2024-09-10T23:09:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a26000640 10 client.25714 handle_command_reply: tid=0 2024-09-10T23:09:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 client.25714 shutdown 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 msgr2=0x557db1f9d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009860 tx=0x7f9a34005d10 comp rx=0 tx=0).stop 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 shutdown_connections 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db1fb5ed0 0x557db1fb82c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1f9b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f9a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/772097070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f9d560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 >> 172.21.15.67:0/772097070 conn(0x557db1f8f120 msgr2=0x557db1f94cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.801+0000 7f9a26000640 0 client.25714 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:15.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.802+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 shutdown_connections 2024-09-10T23:09:15.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.802+0000 7f9a49dc3740 1 -- 172.21.15.67:0/772097070 wait complete. 2024-09-10T23:09:15.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.802+0000 7f9a49dc3740 20 client.25714 trim_cache size 0 max 16384 2024-09-10T23:09:15.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.822+0000 7f9a49dc3740 1 Processor -- start 2024-09-10T23:09:15.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.822+0000 7f9a49dc3740 1 -- start start 2024-09-10T23:09:15.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.822+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.822+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.822+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1fb1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401b0440 con 0x557db1f91c10 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401add10 con 0x557db1f9dba0 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401abb80 con 0x557db1f912f0 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46242/0 (socket says 172.21.15.67:46242) 2024-09-10T23:09:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a3fe00640 1 -- 172.21.15.67:0/3934148127 learned_addr learned my addr 172.21.15.67:0/3934148127 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a45400640 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1fb1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a44a00640 1 -- 172.21.15.67:0/3934148127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 msgr2=0x557db1fb1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1fb1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a44a00640 1 -- 172.21.15.67:0/3934148127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 msgr2=0x557db1f8eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.823+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a44a00640 1 -- 172.21.15.67:0/3934148127 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db1f9d8d0 con 0x557db1f912f0 2024-09-10T23:09:15.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f95e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009b60 tx=0x7f9a34005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a93c7672de7160 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a31200640 1 -- 172.21.15.67:0/3934148127 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3403c070 con 0x557db1f912f0 2024-09-10T23:09:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a31200640 1 -- 172.21.15.67:0/3934148127 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a34011cb0 con 0x557db1f912f0 2024-09-10T23:09:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.824+0000 7f9a31200640 1 -- 172.21.15.67:0/3934148127 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34029070 con 0x557db1f912f0 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3934148127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 msgr2=0x557db1f95e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f95e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009b60 tx=0x7f9a34005f10 comp rx=0 tx=0).stop 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3934148127 shutdown_connections 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db1fb1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f91c10 0x557db1f8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3934148127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f912f0 0x557db1f95e80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.825+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3934148127 >> 172.21.15.67:0/3934148127 conn(0x557db1f8f120 msgr2=0x557db1fb1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.826+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3934148127 shutdown_connections 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.826+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3934148127 wait complete. 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.826+0000 7f9a49dc3740 1 -- start start 2024-09-10T23:09:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.826+0000 7f9a49dc3740 1 Processor -- start 2024-09-10T23:09:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f91c10 0x557db1fb7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db2021a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401ac1d0 con 0x557db1f912f0 2024-09-10T23:09:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401ad640 con 0x557db1f9dba0 2024-09-10T23:09:15.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401ac8a0 con 0x557db1f91c10 2024-09-10T23:09:15.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a49dc3740 10 client.0 _ll_register_callbacks cb 0x7f9a34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:15.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46256/0 (socket says 172.21.15.67:46256) 2024-09-10T23:09:15.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a44a00640 1 -- 172.21.15.67:0/3141079683 learned_addr learned my addr 172.21.15.67:0/3141079683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a45400640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db2021a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.827+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f91c10 0x557db1fb7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 -- 172.21.15.67:0/3141079683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f91c10 msgr2=0x557db1fb7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f91c10 0x557db1fb7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 -- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 msgr2=0x557db2021a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db2021a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 -- 172.21.15.67:0/3141079683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db2021ff0 con 0x557db1f912f0 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a44a00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005ab0 tx=0x7f9a3400f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b60cb87b46e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a26000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3403c070 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9607+0+0 (secure 0 0 0) 0x7f9a340056f0 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.828+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a34011cb0 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9a34029040 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 10 client.16005 fetch_fsmap learned FSMap version 31 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557db20232d0 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a34011cb0 con 0x557db1f912f0 2024-09-10T23:09:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9a340035f0 con 0x557db1f912f0 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 10 client.16005 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 10 client.16005 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 20 client.16005 populate_metadata read hostname 'smithi067' 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db2023a10 0x557db2025e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 4 client.16005 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}] 2024-09-10T23:09:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.829+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}) -- 0x557db1f046a0 con 0x557db2023a10 2024-09-10T23:09:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.830+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db2023a10 0x557db2025e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.831+0000 7f9a3fe00640 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db2023a10 0x557db2025e30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613fc11cbc4113bc server_cookie=43960675cead4a63 in_seq=0 out_seq=0 2024-09-10T23:09:15.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.831+0000 7f9a26000640 10 client.16005 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:15.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.973+0000 7f9a26000640 1 -- 172.21.15.67:0/3141079683 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557db1f046a0 con 0x557db2023a10 2024-09-10T23:09:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.973+0000 7f9a26000640 10 client.16005 handle_command_reply: tid=0 2024-09-10T23:09:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.973+0000 7f9a49dc3740 1 client.16005 shutdown 2024-09-10T23:09:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.973+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 msgr2=0x557db1fb7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.973+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005ab0 tx=0x7f9a3400f650 comp rx=0 tx=0).stop 2024-09-10T23:09:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 shutdown_connections 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557db2023a10 0x557db2025e30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557db1f9dba0 0x557db2021a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557db1f91c10 0x557db1fb7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 --2- 172.21.15.67:0/3141079683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557db1f912f0 0x557db1fb7a00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 >> 172.21.15.67:0/3141079683 conn(0x557db1f8f120 msgr2=0x557db1f8e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a26000640 0 client.16005 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 shutdown_connections 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 1 -- 172.21.15.67:0/3141079683 wait complete. 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.974+0000 7f9a49dc3740 20 client.16005 trim_cache size 0 max 16384 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 30, 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:09:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "22daeea6": { 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 30, 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:09:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 153.4, 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 msgr2=0x7f9a280b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 0x7f9a280b7970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006010 tx=0x7f9a38004a80 comp rx=0 tx=0).stop 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 msgr2=0x7f9a4019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00ea60 tx=0x7f9a2c00ef20 comp rx=0 tx=0).stop 2024-09-10T23:09:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 shutdown_connections 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9a280b54b0 0x7f9a280b7970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a40108f90 0x7f9a401a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a40108610 0x7f9a4019b620 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 --2- 172.21.15.67:0/3425624844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a40102c00 0x7f9a4019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 >> 172.21.15.67:0/3425624844 conn(0x7f9a400fe6d0 msgr2=0x7f9a40104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 shutdown_connections 2024-09-10T23:09:15.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:15.975+0000 7f9a45e00640 1 -- 172.21.15.67:0/3425624844 wait complete. 2024-09-10T23:09:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:16 smithi092 ceph-mon[34223]: pgmap v705: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:16 smithi067 ceph-mon[30939]: pgmap v705: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:16 smithi145 ceph-mon[34397]: pgmap v705: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:09:16.928 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '22daeea6', quiesce took 4.8 seconds. Will release after: 28.599999999999998 2024-09-10T23:09:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:18 smithi092 ceph-mon[34223]: pgmap v706: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:09:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:18 smithi067 ceph-mon[30939]: pgmap v706: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:09:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:18 smithi145 ceph-mon[34397]: pgmap v706: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:09:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:19 smithi092 ceph-mon[34223]: pgmap v707: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:09:19.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:19 smithi092 ceph-mon[34223]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T23:09:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:19 smithi067 ceph-mon[30939]: pgmap v707: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:09:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:19 smithi067 ceph-mon[30939]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T23:09:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:19 smithi145 ceph-mon[34397]: pgmap v707: 129 pgs: 129 active+clean; 4.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:09:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:19 smithi145 ceph-mon[34397]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T23:09:21.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:21 smithi092 ceph-mon[34223]: pgmap v709: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:09:21.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:21 smithi067 ceph-mon[30939]: pgmap v709: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:09:21.839 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:21 smithi145 ceph-mon[34397]: pgmap v709: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:09:23.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:23 smithi092 ceph-mon[34223]: pgmap v710: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:09:23.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:23 smithi067 ceph-mon[30939]: pgmap v710: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:09:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:23 smithi145 ceph-mon[34397]: pgmap v710: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:09:25.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:25 smithi092 ceph-mon[34223]: pgmap v711: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-10T23:09:25.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:25 smithi067 ceph-mon[30939]: pgmap v711: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-10T23:09:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:25 smithi145 ceph-mon[34397]: pgmap v711: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 9 op/s 2024-09-10T23:09:27.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:27 smithi092 ceph-mon[34223]: pgmap v712: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:27.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:27 smithi067 ceph-mon[30939]: pgmap v712: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:27 smithi145 ceph-mon[34397]: pgmap v712: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:29 smithi092 ceph-mon[34223]: pgmap v713: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:29.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:29 smithi067 ceph-mon[30939]: pgmap v713: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:29 smithi145 ceph-mon[34397]: pgmap v713: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 MiB/s wr, 8 op/s 2024-09-10T23:09:31.773 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:31 smithi092 ceph-mon[34223]: pgmap v714: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-09-10T23:09:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:31 smithi067 ceph-mon[30939]: pgmap v714: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-09-10T23:09:31.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:31 smithi145 ceph-mon[34397]: pgmap v714: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-09-10T23:09:33.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:33 smithi067 ceph-mon[30939]: pgmap v715: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 656 KiB/s wr, 6 op/s 2024-09-10T23:09:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:33 smithi145 ceph-mon[34397]: pgmap v715: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 656 KiB/s wr, 6 op/s 2024-09-10T23:09:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:33 smithi092 ceph-mon[34223]: pgmap v715: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 656 KiB/s wr, 6 op/s 2024-09-10T23:09:35.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:35 smithi067 ceph-mon[30939]: pgmap v716: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 656 KiB/s wr, 5 op/s 2024-09-10T23:09:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:35 smithi145 ceph-mon[34397]: pgmap v716: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 656 KiB/s wr, 5 op/s 2024-09-10T23:09:36.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:35 smithi092 ceph-mon[34223]: pgmap v716: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 656 KiB/s wr, 5 op/s 2024-09-10T23:09:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:37 smithi067 ceph-mon[30939]: pgmap v717: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-10T23:09:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:37 smithi145 ceph-mon[34397]: pgmap v717: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-10T23:09:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:37 smithi092 ceph-mon[34223]: pgmap v717: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-10T23:09:39.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:39 smithi067 ceph-mon[30939]: pgmap v718: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:39.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:39 smithi145 ceph-mon[34397]: pgmap v718: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:39 smithi092 ceph-mon[34223]: pgmap v718: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:41.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:41 smithi067 ceph-mon[30939]: pgmap v719: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:09:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:41 smithi145 ceph-mon[34397]: pgmap v719: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:09:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:41 smithi092 ceph-mon[34223]: pgmap v719: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:09:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:43 smithi067 ceph-mon[30939]: pgmap v720: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:09:43.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:43 smithi145 ceph-mon[34397]: pgmap v720: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:09:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:43 smithi092 ceph-mon[34223]: pgmap v720: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:09:45.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:09:45.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:45 smithi145 ceph-mon[34397]: pgmap v721: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:09:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:45 smithi092 ceph-mon[34223]: pgmap v721: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:09:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:45 smithi067 ceph-mon[30939]: pgmap v721: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:09:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 -- 172.21.15.67:0/866190196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d810b320 msgr2=0x7f35d8111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 --2- 172.21.15.67:0/866190196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d810b320 0x7f35d8111b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009ca0 tx=0x7f35d402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 -- 172.21.15.67:0/866190196 shutdown_connections 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 --2- 172.21.15.67:0/866190196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d810b320 0x7f35d8111b20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 --2- 172.21.15.67:0/866190196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810a9a0 0x7f35d810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 --2- 172.21.15.67:0/866190196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d8104f90 0x7f35d8105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 -- 172.21.15.67:0/866190196 >> 172.21.15.67:0/866190196 conn(0x7f35d8100a00 msgr2=0x7f35d8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 -- 172.21.15.67:0/866190196 shutdown_connections 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.647+0000 7f35e0800640 1 -- 172.21.15.67:0/866190196 wait complete. 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 Processor -- start 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 -- start start 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 0x7f35d819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 0x7f35d81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8114e50 con 0x7f35d8104f90 2024-09-10T23:09:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8114cd0 con 0x7f35d810b320 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.648+0000 7f35e0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8114fd0 con 0x7f35d810a9a0 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44532/0 (socket says 172.21.15.67:44532) 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 -- 172.21.15.67:0/3906488452 learned_addr learned my addr 172.21.15.67:0/3906488452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dd400640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 0x7f35d819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35de800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 0x7f35d81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 -- 172.21.15.67:0/3906488452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 msgr2=0x7f35d819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 0x7f35d819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 -- 172.21.15.67:0/3906488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 msgr2=0x7f35d81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 0x7f35d81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dde00640 1 -- 172.21.15.67:0/3906488452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d81a13f0 con 0x7f35d8104f90 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35dd400640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 0x7f35d819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.649+0000 7f35de800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 0x7f35d81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35dde00640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d770 tx=0x7f35cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc017070 con 0x7f35d8104f90 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35cc0054b0 con 0x7f35d8104f90 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d81a16e0 con 0x7f35d8104f90 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc004c60 con 0x7f35d8104f90 2024-09-10T23:09:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.650+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81a7da0 con 0x7f35d8104f90 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.651+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8105370 con 0x7f35d8104f90 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.653+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f35cc004e00 con 0x7f35d8104f90 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.655+0000 7f35d2a00640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 0x7f35c00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.655+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9519+0+0 (secure 0 0 0) 0x7f35cc0e62d0 con 0x7f35d8104f90 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.655+0000 7f35dd400640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 0x7f35c00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.656+0000 7f35dd400640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 0x7f35c00b7990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35c80058c0 tx=0x7f35c8005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.657+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f35cc0aca90 con 0x7f35d8104f90 2024-09-10T23:09:46.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.889+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35d802e5c0 con 0x7f35d8104f90 2024-09-10T23:09:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.890+0000 7f35d2a00640 1 -- 172.21.15.67:0/3906488452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f35cc0ac390 con 0x7f35d8104f90 2024-09-10T23:09:46.893 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:09:46.893 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:09:46.894 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 msgr2=0x7f35c00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 0x7f35c00b7990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35c80058c0 tx=0x7f35c8005850 comp rx=0 tx=0).stop 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 msgr2=0x7f35d819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d770 tx=0x7f35cc00dc30 comp rx=0 tx=0).stop 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 shutdown_connections 2024-09-10T23:09:46.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f35c00b54d0 0x7f35c00b7990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35d810b320 0x7f35d81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35d810a9a0 0x7f35d819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 --2- 172.21.15.67:0/3906488452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35d8104f90 0x7f35d819b340 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.892+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 >> 172.21.15.67:0/3906488452 conn(0x7f35d8100a00 msgr2=0x7f35d8101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.893+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 shutdown_connections 2024-09-10T23:09:46.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:46.893+0000 7f35e0800640 1 -- 172.21.15.67:0/3906488452 wait complete. 2024-09-10T23:09:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:47 smithi067 ceph-mon[30939]: pgmap v722: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:09:47.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:47 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3906488452' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:47 smithi145 ceph-mon[34397]: pgmap v722: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:09:47.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:47 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3906488452' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:47 smithi092 ceph-mon[34223]: pgmap v722: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:09:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:47 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3906488452' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:48.037 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 22daeea6 --release --await' 2024-09-10T23:09:48.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 22daeea6 --release --await 2024-09-10T23:09:48.436 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 -- 172.21.15.67:0/1016317606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 msgr2=0x7f5d10105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 --2- 172.21.15.67:0/1016317606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d10105380 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402a410 comp rx=0 tx=0).stop 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 -- 172.21.15.67:0/1016317606 shutdown_connections 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 --2- 172.21.15.67:0/1016317606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010b330 0x7f5d10111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 --2- 172.21.15.67:0/1016317606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010a9b0 0x7f5d1010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 --2- 172.21.15.67:0/1016317606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d10105380 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 -- 172.21.15.67:0/1016317606 >> 172.21.15.67:0/1016317606 conn(0x7f5d10100a50 msgr2=0x7f5d10102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 -- 172.21.15.67:0/1016317606 shutdown_connections 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.069+0000 7f5d18000640 1 -- 172.21.15.67:0/1016317606 wait complete. 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.070+0000 7f5d18000640 1 Processor -- start 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.070+0000 7f5d18000640 1 -- start start 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010a9b0 0x7f5d1019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010b330 0x7f5d101a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10114df0 con 0x7f5d10104fa0 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10114c70 con 0x7f5d1010a9b0 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d18000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10114f70 con 0x7f5d1010b330 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d15600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d15600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55576/0 (socket says 172.21.15.67:55576) 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d15600640 1 -- 172.21.15.67:0/2049337387 learned_addr learned my addr 172.21.15.67:0/2049337387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d14c00640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010a9b0 0x7f5d1019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.071+0000 7f5d16000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010b330 0x7f5d101a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 -- 172.21.15.67:0/2049337387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010b330 msgr2=0x7f5d101a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010b330 0x7f5d101a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 -- 172.21.15.67:0/2049337387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010a9b0 msgr2=0x7f5d1019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010a9b0 0x7f5d1019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 -- 172.21.15.67:0/2049337387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d101a14f0 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d15600640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009230 tx=0x7f5d04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d0e000640 1 -- 172.21.15.67:0/2049337387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0403f030 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101a1780 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d0e000640 1 -- 172.21.15.67:0/2049337387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d04003a80 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.072+0000 7f5d0e000640 1 -- 172.21.15.67:0/2049337387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04035070 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.073+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a7ea0 con 0x7f5d10104fa0 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 Processor -- start 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 -- start start 2024-09-10T23:09:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d0e000640 1 -- 172.21.15.67:0/2049337387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5d0403bb90 con 0x7f5d10104fa0 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27398360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561b27303290 con 0x561b2738bff0 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561b272f26c0 con 0x561b2738c910 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.076+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561b272f64d0 con 0x561b273988a0 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55588/0 (socket says 172.21.15.67:55588) 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d15600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27398360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 -- 172.21.15.67:0/2881723585 learned_addr learned my addr 172.21.15.67:0/2881723585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d14c00640 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 -- 172.21.15.67:0/2881723585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 msgr2=0x561b2739ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 -- 172.21.15.67:0/2881723585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 msgr2=0x561b27398360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27398360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.077+0000 7f5d16000640 1 -- 172.21.15.67:0/2881723585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2727f460 con 0x561b2738bff0 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.078+0000 7f5d16000640 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009f70 tx=0x7f5d08010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a107cc9689f4d95f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.078+0000 7f5d02600640 1 -- 172.21.15.67:0/2881723585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003b70 con 0x561b2738bff0 2024-09-10T23:09:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.078+0000 7f5d02600640 1 -- 172.21.15.67:0/2881723585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d08003d10 con 0x561b2738bff0 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.078+0000 7f5d02600640 1 -- 172.21.15.67:0/2881723585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d080118c0 con 0x561b2738bff0 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.078+0000 7f5d0e000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 0x7f5cf80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2881723585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 msgr2=0x561b2738c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009f70 tx=0x7f5d08010860 comp rx=0 tx=0).stop 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d0e000640 1 -- 172.21.15.67:0/2049337387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9519+0+0 (secure 0 0 0) 0x7f5d04106d70 con 0x7f5d10104fa0 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2881723585 shutdown_connections 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d15600640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 0x7f5cf80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27398360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2881723585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738c3d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2881723585 >> 172.21.15.67:0/2881723585 conn(0x561b27389e20 msgr2=0x561b2738afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2881723585 shutdown_connections 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.079+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2881723585 wait complete. 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.080+0000 7f5d1aa7e740 1 -- start start 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.080+0000 7f5d15600640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 0x7f5cf80b7990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x561b2738f170 tx=0x7f5d0410a820 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.080+0000 7f5d1aa7e740 1 Processor -- start 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 0x561b273958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b273988a0 0x561b27396340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561b27153bb0 con 0x561b273988a0 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561b2728d6c0 con 0x561b2738bff0 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561b2739b550 con 0x561b2738c910 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d1aa7e740 10 client.0 _ll_register_callbacks cb 0x561b272a8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:49.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d14c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b273988a0 0x561b27396340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d14c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:49372/0 (socket says 172.21.15.67:49372) 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d14c00640 1 -- 172.21.15.67:0/3455929713 learned_addr learned my addr 172.21.15.67:0/3455929713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.081+0000 7f5d16000640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 0x561b273958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 -- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 msgr2=0x561b273958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 0x561b273958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 -- 172.21.15.67:0/3455929713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b273988a0 msgr2=0x561b27396340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b273988a0 0x561b27396340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 -- 172.21.15.67:0/3455929713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b27050b00 con 0x561b2738c910 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d16000640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 0x561b273958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d14c00640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00bad0 tx=0x7f5cfc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d37386b13ee3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5cf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:09:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003e20 con 0x561b2738c910 2024-09-10T23:09:49.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d08003d10 con 0x561b2738c910 2024-09-10T23:09:49.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.082+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9519+0+0 (secure 0 0 0) 0x7f5cfc011440 con 0x561b2738c910 2024-09-10T23:09:49.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.083+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc005020 con 0x561b2738c910 2024-09-10T23:09:49.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.083+0000 7f5d1aa7e740 10 client.25738 fetch_fsmap learned FSMap version 31 2024-09-10T23:09:49.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.083+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b270c7960 con 0x561b2738c910 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.083+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d08003d10 con 0x561b2738c910 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5cfc011640 con 0x561b2738c910 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 10 client.25738 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 10 client.25738 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 20 client.25738 populate_metadata read hostname 'smithi067' 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b273a8540 0x561b273aa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 4 client.25738 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b273051e0 con 0x561b273a8540 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.084+0000 7f5d16000640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b273a8540 0x561b273aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.085+0000 7f5d16000640 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b273a8540 0x561b273aa930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8734be0c128169b3 server_cookie=3901c5e562c8629d in_seq=0 out_seq=0 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.085+0000 7f5cf6a00640 10 client.25738 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5cf6a00640 1 -- 172.21.15.67:0/3455929713 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b273051e0 con 0x561b273a8540 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5cf6a00640 10 client.25738 handle_command_reply: tid=0 2024-09-10T23:09:49.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 client.25738 shutdown 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 msgr2=0x561b27395e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00bad0 tx=0x7f5cfc00bf90 comp rx=0 tx=0).stop 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 shutdown_connections 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b273a8540 0x561b273aa930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b273988a0 0x561b27396340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b2738c910 0x561b27395e00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5cf6a00640 0 client.25738 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/3455929713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738bff0 0x561b273958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 >> 172.21.15.67:0/3455929713 conn(0x561b27389e20 msgr2=0x561b2738f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 shutdown_connections 2024-09-10T23:09:49.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/3455929713 wait complete. 2024-09-10T23:09:49.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.086+0000 7f5d1aa7e740 20 client.25738 trim_cache size 0 max 16384 2024-09-10T23:09:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 Processor -- start 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 -- start start 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27392ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b27393010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b4a10 con 0x561b2738bff0 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b22e0 con 0x561b2738c910 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b0150 con 0x561b273988a0 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.105+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27392ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55612/0 (socket says 172.21.15.67:55612) 2024-09-10T23:09:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d16000640 1 -- 172.21.15.67:0/4263864131 learned_addr learned my addr 172.21.15.67:0/4263864131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d16000640 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b27393010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 -- 172.21.15.67:0/4263864131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 msgr2=0x561b27393010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b27393010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 -- 172.21.15.67:0/4263864131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 msgr2=0x561b2738b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 -- 172.21.15.67:0/4263864131 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2738a8a0 con 0x561b2738c910 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.106+0000 7f5d14c00640 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27392ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc011ca0 tx=0x7f5cfc004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfe5db5817bb7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d01c00640 1 -- 172.21.15.67:0/4263864131 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011440 con 0x561b2738c910 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d01c00640 1 -- 172.21.15.67:0/4263864131 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cfc0115e0 con 0x561b2738c910 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d01c00640 1 -- 172.21.15.67:0/4263864131 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004b90 con 0x561b2738c910 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/4263864131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 msgr2=0x561b27392ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27392ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc011ca0 tx=0x7f5cfc004490 comp rx=0 tx=0).stop 2024-09-10T23:09:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/4263864131 shutdown_connections 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b27393010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b27392ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/4263864131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b2738b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.107+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/4263864131 >> 172.21.15.67:0/4263864131 conn(0x561b27389e20 msgr2=0x561b27388db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.108+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/4263864131 shutdown_connections 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.108+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/4263864131 wait complete. 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.108+0000 7f5d1aa7e740 1 -- start start 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.108+0000 7f5d1aa7e740 1 Processor -- start 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b2739d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b07a0 con 0x561b2738bff0 2024-09-10T23:09:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b1c10 con 0x561b2738c910 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b0e70 con 0x561b273988a0 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d1aa7e740 10 client.0 _ll_register_callbacks cb 0x7f5cfc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55624/0 (socket says 172.21.15.67:55624) 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d16000640 1 -- 172.21.15.67:0/2964607092 learned_addr learned my addr 172.21.15.67:0/2964607092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d16000640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.109+0000 7f5d14c00640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b2739d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 -- 172.21.15.67:0/2964607092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 msgr2=0x561b2739d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 -- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 msgr2=0x561b2739d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b2739d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 -- 172.21.15.67:0/2964607092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b273a70d0 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d16000640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5d08011600 tx=0x7f5d08013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cb29f5c33461b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5cf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08024040 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc0115e0 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.110+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9519+0+0 (secure 0 0 0) 0x7f5d08011d40 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5d08040050 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 10 client.16029 fetch_fsmap learned FSMap version 31 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b27416080 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cfc0115e0 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5d0802a050 con 0x561b2738bff0 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 10 client.16029 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 10 client.16029 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:09:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 20 client.16029 populate_metadata read hostname 'smithi067' 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b27416600 0x561b274189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 4 client.16029 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "22daeea6", "release": true, "await": true}] 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.111+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "22daeea6", "release": true, "await": true}) -- 0x561b27300c90 con 0x561b27416600 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.112+0000 7f5d14c00640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b27416600 0x561b274189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.112+0000 7f5d14c00640 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b27416600 0x561b274189f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff07a87ea38be61 server_cookie=d650404d041d6994 in_seq=0 out_seq=0 2024-09-10T23:09:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.112+0000 7f5cf6a00640 10 client.16029 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:49.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5cf6a00640 1 -- 172.21.15.67:0/2964607092 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561b27300c90 con 0x561b27416600 2024-09-10T23:09:49.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5cf6a00640 10 client.16029 handle_command_reply: tid=0 2024-09-10T23:09:49.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 client.16029 shutdown 2024-09-10T23:09:49.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 msgr2=0x561b273a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5d08011600 tx=0x7f5d08013040 comp rx=0 tx=0).stop 2024-09-10T23:09:49.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 shutdown_connections 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x561b27416600 0x561b274189f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b273988a0 0x561b2739d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561b2738c910 0x561b2739d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 --2- 172.21.15.67:0/2964607092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b2738bff0 0x561b273a0dd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 >> 172.21.15.67:0/2964607092 conn(0x561b27389e20 msgr2=0x561b27396ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5cf6a00640 0 client.16029 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 shutdown_connections 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.136+0000 7f5d1aa7e740 1 -- 172.21.15.67:0/2964607092 wait complete. 2024-09-10T23:09:49.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.137+0000 7f5d1aa7e740 20 client.16029 trim_cache size 0 max 16384 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 32, 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "22daeea6": { 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 32, 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:09:49.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 153.4, 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:49.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:49.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:09:49.142 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:09:49.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.137+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 msgr2=0x7f5cf80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.137+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 0x7f5cf80b7990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x561b2738f170 tx=0x7f5d0410a820 comp rx=0 tx=0).stop 2024-09-10T23:09:49.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.137+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 msgr2=0x7f5d1019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.137+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009230 tx=0x7f5d04004270 comp rx=0 tx=0).stop 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 shutdown_connections 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f5cf80b54d0 0x7f5cf80b7990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d1010b330 0x7f5d101a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d1010a9b0 0x7f5d1019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 --2- 172.21.15.67:0/2049337387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d10104fa0 0x7f5d1019b440 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 >> 172.21.15.67:0/2049337387 conn(0x7f5d10100a50 msgr2=0x7f5d10101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 shutdown_connections 2024-09-10T23:09:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:09:49.138+0000 7f5d18000640 1 -- 172.21.15.67:0/2049337387 wait complete. 2024-09-10T23:09:49.823 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:49 smithi092 ceph-mon[34223]: pgmap v723: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:49.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:49 smithi145 ceph-mon[34397]: pgmap v723: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:49.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:49 smithi067 ceph-mon[30939]: pgmap v723: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:09:50.017 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '22daeea6', total seconds elapsed: 37.9 2024-09-10T23:09:51.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:51 smithi145 ceph-mon[34397]: pgmap v724: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-09-10T23:09:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:51 smithi092 ceph-mon[34223]: pgmap v724: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-09-10T23:09:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:51 smithi067 ceph-mon[30939]: pgmap v724: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-09-10T23:09:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:53 smithi145 ceph-mon[34397]: pgmap v725: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:09:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:53.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:09:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:53 smithi092 ceph-mon[34223]: pgmap v725: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:09:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:54.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:09:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:53 smithi067 ceph-mon[30939]: pgmap v725: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:09:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:09:55.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:55 smithi145 ceph-mon[34397]: pgmap v726: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 MiB/s wr, 7 op/s 2024-09-10T23:09:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:55 smithi092 ceph-mon[34223]: pgmap v726: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 MiB/s wr, 7 op/s 2024-09-10T23:09:56.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:55 smithi067 ceph-mon[30939]: pgmap v726: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 MiB/s wr, 7 op/s 2024-09-10T23:09:57.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:57 smithi145 ceph-mon[34397]: pgmap v727: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 9 op/s 2024-09-10T23:09:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:57 smithi092 ceph-mon[34223]: pgmap v727: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 9 op/s 2024-09-10T23:09:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:57 smithi067 ceph-mon[30939]: pgmap v727: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 9 op/s 2024-09-10T23:09:59.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:09:59 smithi145 ceph-mon[34397]: pgmap v728: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-10T23:10:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:09:59 smithi092 ceph-mon[34223]: pgmap v728: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-10T23:10:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:09:59 smithi067 ceph-mon[30939]: pgmap v728: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 7 op/s 2024-09-10T23:10:00.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:00 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T23:10:01.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:00 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T23:10:01.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:00 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T23:10:01.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:01 smithi145 ceph-mon[34397]: pgmap v729: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:10:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:01 smithi092 ceph-mon[34223]: pgmap v729: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:10:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:01 smithi067 ceph-mon[30939]: pgmap v729: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:10:03.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:03 smithi145 ceph-mon[34397]: pgmap v730: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:10:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:03 smithi092 ceph-mon[34223]: pgmap v730: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:10:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:03 smithi067 ceph-mon[30939]: pgmap v730: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-09-10T23:10:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:05 smithi145 ceph-mon[34397]: pgmap v731: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-09-10T23:10:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:05 smithi092 ceph-mon[34223]: pgmap v731: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-09-10T23:10:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:05 smithi067 ceph-mon[30939]: pgmap v731: 129 pgs: 129 active+clean; 5.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-09-10T23:10:07.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:07 smithi145 ceph-mon[34397]: pgmap v732: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:10:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:07 smithi092 ceph-mon[34223]: pgmap v732: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:10:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:07 smithi067 ceph-mon[30939]: pgmap v732: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-09-10T23:10:08.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:08 smithi145 ceph-mon[34397]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T23:10:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:08 smithi092 ceph-mon[34223]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T23:10:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:08 smithi067 ceph-mon[30939]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T23:10:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:09 smithi145 ceph-mon[34397]: pgmap v734: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:10:09.949 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:09 smithi067 ceph-mon[30939]: pgmap v734: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:10:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:09 smithi092 ceph-mon[34223]: pgmap v734: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:10:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:11 smithi145 ceph-mon[34397]: pgmap v735: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:10:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:11 smithi092 ceph-mon[34223]: pgmap v735: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:10:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:11 smithi067 ceph-mon[30939]: pgmap v735: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:10:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:13 smithi145 ceph-mon[34397]: pgmap v736: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:10:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:13 smithi145 ceph-mon[34397]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T23:10:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:13 smithi092 ceph-mon[34223]: pgmap v736: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:10:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:13 smithi092 ceph-mon[34223]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T23:10:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:13 smithi067 ceph-mon[30939]: pgmap v736: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:10:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:13 smithi067 ceph-mon[30939]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T23:10:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:15 smithi145 ceph-mon[34397]: pgmap v738: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:10:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:15 smithi092 ceph-mon[34223]: pgmap v738: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:10:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:15 smithi067 ceph-mon[30939]: pgmap v738: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:10:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:17 smithi145 ceph-mon[34397]: pgmap v739: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:17 smithi092 ceph-mon[34223]: pgmap v739: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:17 smithi067 ceph-mon[30939]: pgmap v739: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:19 smithi092 ceph-mon[34223]: pgmap v740: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:19 smithi067 ceph-mon[30939]: pgmap v740: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:19 smithi145 ceph-mon[34397]: pgmap v740: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:21 smithi092 ceph-mon[34223]: pgmap v741: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:21 smithi067 ceph-mon[30939]: pgmap v741: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:21 smithi145 ceph-mon[34397]: pgmap v741: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:23 smithi092 ceph-mon[34223]: pgmap v742: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:23 smithi067 ceph-mon[30939]: pgmap v742: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:23 smithi145 ceph-mon[34397]: pgmap v742: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:25 smithi092 ceph-mon[34223]: pgmap v743: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:25 smithi067 ceph-mon[30939]: pgmap v743: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:25 smithi145 ceph-mon[34397]: pgmap v743: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:27 smithi092 ceph-mon[34223]: pgmap v744: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:27 smithi067 ceph-mon[30939]: pgmap v744: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:27 smithi145 ceph-mon[34397]: pgmap v744: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:29 smithi092 ceph-mon[34223]: pgmap v745: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:29 smithi067 ceph-mon[30939]: pgmap v745: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:29 smithi145 ceph-mon[34397]: pgmap v745: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:31 smithi092 ceph-mon[34223]: pgmap v746: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:31 smithi067 ceph-mon[30939]: pgmap v746: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:31 smithi145 ceph-mon[34397]: pgmap v746: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:10:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:33 smithi092 ceph-mon[34223]: pgmap v747: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:33 smithi067 ceph-mon[30939]: pgmap v747: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:34.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:33 smithi145 ceph-mon[34397]: pgmap v747: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:35 smithi092 ceph-mon[34223]: pgmap v748: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:10:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:35 smithi067 ceph-mon[30939]: pgmap v748: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:10:36.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:35 smithi145 ceph-mon[34397]: pgmap v748: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:10:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:37 smithi092 ceph-mon[34223]: pgmap v749: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:37 smithi067 ceph-mon[30939]: pgmap v749: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:37 smithi145 ceph-mon[34397]: pgmap v749: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:39.932 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:39 smithi145 ceph-mon[34397]: pgmap v750: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:10:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:39 smithi092 ceph-mon[34223]: pgmap v750: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:10:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:39 smithi067 ceph-mon[30939]: pgmap v750: 129 pgs: 129 active+clean; 5.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:10:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:41 smithi092 ceph-mon[34223]: pgmap v751: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:41 smithi067 ceph-mon[30939]: pgmap v751: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:41 smithi145 ceph-mon[34397]: pgmap v751: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:10:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:43 smithi092 ceph-mon[34223]: pgmap v752: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:10:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:43 smithi067 ceph-mon[30939]: pgmap v752: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:10:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:43 smithi145 ceph-mon[34397]: pgmap v752: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:10:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:45 smithi092 ceph-mon[34223]: pgmap v753: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:10:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:45 smithi067 ceph-mon[30939]: pgmap v753: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:10:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:45 smithi145 ceph-mon[34397]: pgmap v753: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:10:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:47 smithi092 ceph-mon[34223]: pgmap v754: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:47 smithi067 ceph-mon[30939]: pgmap v754: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:47 smithi145 ceph-mon[34397]: pgmap v754: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:10:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:49 smithi092 ceph-mon[34223]: pgmap v755: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:49 smithi067 ceph-mon[30939]: pgmap v755: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:49 smithi145 ceph-mon[34397]: pgmap v755: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:10:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:51 smithi092 ceph-mon[34223]: pgmap v756: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:10:52.047 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:10:52.047 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-10T23:10:52.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:10:52.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:51 smithi067 ceph-mon[30939]: pgmap v756: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:10:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:51 smithi145 ceph-mon[34397]: pgmap v756: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:10:52.500 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 -- 172.21.15.67:0/1375006088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3464075de0 msgr2=0x7f346410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 --2- 172.21.15.67:0/1375006088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3464075de0 0x7f346410ece0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3450009900 tx=0x7f345002a410 comp rx=0 tx=0).stop 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 -- 172.21.15.67:0/1375006088 shutdown_connections 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 --2- 172.21.15.67:0/1375006088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f346410f220 0x7f3464111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 --2- 172.21.15.67:0/1375006088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3464075de0 0x7f346410ece0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 --2- 172.21.15.67:0/1375006088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34640754c0 0x7f34640758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 -- 172.21.15.67:0/1375006088 >> 172.21.15.67:0/1375006088 conn(0x7f34640fe660 msgr2=0x7f3464100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 -- 172.21.15.67:0/1375006088 shutdown_connections 2024-09-10T23:10:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.252+0000 7f3463e00640 1 -- 172.21.15.67:0/1375006088 wait complete. 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.253+0000 7f3463e00640 1 Processor -- start 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 -- start start 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3464075de0 0x7f346419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346410f220 0x7f34641a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3464114cb0 con 0x7f34640754c0 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3464114b30 con 0x7f346410f220 2024-09-10T23:10:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3463e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3464114e30 con 0x7f3464075de0 2024-09-10T23:10:53.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3462a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:53.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.254+0000 7f3462a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51146/0 (socket says 172.21.15.67:51146) 2024-09-10T23:10:53.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 -- 172.21.15.67:0/3505040276 learned_addr learned my addr 172.21.15.67:0/3505040276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:53.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462000640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3464075de0 0x7f346419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:53.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3463400640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346410f220 0x7f34641a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 -- 172.21.15.67:0/3505040276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3464075de0 msgr2=0x7f346419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3464075de0 0x7f346419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 -- 172.21.15.67:0/3505040276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346410f220 msgr2=0x7f34641a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346410f220 0x7f34641a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 -- 172.21.15.67:0/3505040276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34641a12b0 con 0x7f34640754c0 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.255+0000 7f3462a00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:53.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.256+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400cd80 con 0x7f34640754c0 2024-09-10T23:10:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.256+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34540108c0 con 0x7f34640754c0 2024-09-10T23:10:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.256+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540187c0 con 0x7f34640754c0 2024-09-10T23:10:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.256+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34641a15a0 con 0x7f34640754c0 2024-09-10T23:10:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.256+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34641a7e30 con 0x7f34640754c0 2024-09-10T23:10:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.257+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34640769d0 con 0x7f34640754c0 2024-09-10T23:10:53.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.260+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3454010430 con 0x7f34640754c0 2024-09-10T23:10:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.262+0000 7f345b400640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 0x7f34480b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.263+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f34540e6700 con 0x7f34640754c0 2024-09-10T23:10:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.263+0000 7f3462000640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 0x7f34480b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.263+0000 7f3462000640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 0x7f34480b7a40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34500057c0 tx=0x7f34500058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:53.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.266+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f34540ac7f0 con 0x7f34640754c0 2024-09-10T23:10:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.531+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346410a3d0 con 0x7f34640754c0 2024-09-10T23:10:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.532+0000 7f345b400640 1 -- 172.21.15.67:0/3505040276 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f34540b1220 con 0x7f34640754c0 2024-09-10T23:10:53.533 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:10:53.533 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:10:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:10:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 msgr2=0x7f34480b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 0x7f34480b7a40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34500057c0 tx=0x7f34500058d0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 msgr2=0x7f346419b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 shutdown_connections 2024-09-10T23:10:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f34480b5580 0x7f34480b7a40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346410f220 0x7f34641a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3464075de0 0x7f346419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 --2- 172.21.15.67:0/3505040276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640754c0 0x7f346419b200 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 >> 172.21.15.67:0/3505040276 conn(0x7f34640fe660 msgr2=0x7f34640ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.535+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 shutdown_connections 2024-09-10T23:10:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:53.536+0000 7f3463e00640 1 -- 172.21.15.67:0/3505040276 wait complete. 2024-09-10T23:10:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:53 smithi067 ceph-mon[30939]: pgmap v757: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:53.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:53 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3505040276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:53 smithi092 ceph-mon[34223]: pgmap v757: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:53 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3505040276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:53 smithi145 ceph-mon[34397]: pgmap v757: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:10:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:53 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3505040276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:54.586 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-10T23:10:54.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-10T23:10:55.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:54 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:55.030 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:55.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:54 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:55.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:54 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 -- 172.21.15.67:0/958482893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 msgr2=0x7f49c8111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 --2- 172.21.15.67:0/958482893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c8111640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009be0 tx=0x7f49c002a750 comp rx=0 tx=0).stop 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 -- 172.21.15.67:0/958482893 shutdown_connections 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 --2- 172.21.15.67:0/958482893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c8111640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 --2- 172.21.15.67:0/958482893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c8101fc0 0x7f49c810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 --2- 172.21.15.67:0/958482893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c81016a0 0x7f49c8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 -- 172.21.15.67:0/958482893 >> 172.21.15.67:0/958482893 conn(0x7f49c80fd5b0 msgr2=0x7f49c80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.756+0000 7f49c7200640 1 -- 172.21.15.67:0/958482893 shutdown_connections 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.757+0000 7f49c7200640 1 -- 172.21.15.67:0/958482893 wait complete. 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.757+0000 7f49c7200640 1 Processor -- start 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 -- start start 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c8101fc0 0x7f49c819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8114c80 con 0x7f49c81016a0 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8114b00 con 0x7f49c810f250 2024-09-10T23:10:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8114e00 con 0x7f49c8101fc0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51166/0 (socket says 172.21.15.67:51166) 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c5e00640 1 -- 172.21.15.67:0/2328859482 learned_addr learned my addr 172.21.15.67:0/2328859482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.758+0000 7f49c6800640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49bfe00640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c8101fc0 0x7f49c819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 -- 172.21.15.67:0/2328859482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c8101fc0 msgr2=0x7f49c819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c8101fc0 0x7f49c819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 -- 172.21.15.67:0/2328859482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 msgr2=0x7f49c81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 -- 172.21.15.67:0/2328859482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c81a1180 con 0x7f49c81016a0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.759+0000 7f49c5e00640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9f0 tx=0x7f49b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.760+0000 7f49bea00640 1 -- 172.21.15.67:0/2328859482 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800ccc0 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.760+0000 7f49bea00640 1 -- 172.21.15.67:0/2328859482 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49b800ce60 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.760+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c81a1470 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.760+0000 7f49bea00640 1 -- 172.21.15.67:0/2328859482 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8010430 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.760+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c81a7b30 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.764+0000 7f49bea00640 1 -- 172.21.15.67:0/2328859482 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f49b8004f70 con 0x7f49c81016a0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.764+0000 7f49cf49f740 1 Processor -- start 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.764+0000 7f49cf49f740 1 -- start start 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d92f0 0x562a029d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 0x562a029e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562a0294eca0 con 0x562a029d9c10 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a0293e0d0 con 0x562a029e5ba0 2024-09-10T23:10:55.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49cf49f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562a02941ee0 con 0x562a029d92f0 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49bfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 0x562a029e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49bfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 0x562a029e7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47640/0 (socket says 172.21.15.67:47640) 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51178/0 (socket says 172.21.15.67:51178) 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49bfe00640 1 -- 172.21.15.67:0/232871608 learned_addr learned my addr 172.21.15.67:0/232871608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.765+0000 7f49c6800640 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d92f0 0x562a029d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c5e00640 1 -- 172.21.15.67:0/232871608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d92f0 msgr2=0x562a029d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c5e00640 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d92f0 0x562a029d96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c5e00640 1 -- 172.21.15.67:0/232871608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 msgr2=0x562a029e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c5e00640 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 0x562a029e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c5e00640 1 -- 172.21.15.67:0/232871608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a028f0400 con 0x562a029d9c10 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49bea00640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 0x7f49ac0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.766+0000 7f49c6800640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 0x7f49ac0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.767+0000 7f49bea00640 1 -- 172.21.15.67:0/2328859482 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f49b80e6770 con 0x7f49c81016a0 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.767+0000 7f49c5e00640 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49b80e3a80 tx=0x7f49b80ecb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515f1ddb31c70c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.767+0000 7f49b7000640 1 -- 172.21.15.67:0/232871608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a8650 con 0x562a029d9c10 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.767+0000 7f49b7000640 1 -- 172.21.15.67:0/232871608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49b80ac2c0 con 0x562a029d9c10 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.767+0000 7f49b7000640 1 -- 172.21.15.67:0/232871608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80abf20 con 0x562a029d9c10 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 -- 172.21.15.67:0/232871608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 msgr2=0x562a029e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49c6800640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 0x7f49ac0b7a40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006010 tx=0x7f49c00098c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49b80e3a80 tx=0x7f49b80ecb20 comp rx=0 tx=0).stop 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 -- 172.21.15.67:0/232871608 shutdown_connections 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029e5ba0 0x562a029e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d9c10 0x562a029e5660 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 --2- 172.21.15.67:0/232871608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d92f0 0x562a029d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 -- 172.21.15.67:0/232871608 >> 172.21.15.67:0/232871608 conn(0x562a029d7120 msgr2=0x562a029d82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 -- 172.21.15.67:0/232871608 shutdown_connections 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.768+0000 7f49cf49f740 1 -- 172.21.15.67:0/232871608 wait complete. 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.769+0000 7f49cf49f740 1 -- start start 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.769+0000 7f49cf49f740 1 Processor -- start 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d92f0 0x562a029e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 0x562a029e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 0x562a029e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562a0279f7a0 con 0x562a029e5ba0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a028e3a10 con 0x562a029d92f0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562a029e8820 con 0x562a029d9c10 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49cf49f740 10 client.0 _ll_register_callbacks cb 0x562a028f3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49c6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 0x562a029e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49bfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 0x562a029e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d92f0 0x562a029e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49c6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 0x562a029e2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51194/0 (socket says 172.21.15.67:51194) 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.770+0000 7f49c6800640 1 -- 172.21.15.67:0/3371648736 learned_addr learned my addr 172.21.15.67:0/3371648736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 -- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d92f0 msgr2=0x562a029e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d92f0 0x562a029e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 -- 172.21.15.67:0/3371648736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 msgr2=0x562a029e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 0x562a029e2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 -- 172.21.15.67:0/3371648736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a026abb90 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49bfe00640 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 0x562a029e2920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49b00126d0 tx=0x7f49b0012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3ff7080fe7d626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.771+0000 7f49ab400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0004050 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f49b00041f0 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b80ac2c0 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f49b0025050 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49cf49f740 10 client.25762 fetch_fsmap learned FSMap version 31 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a029f4220 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.772+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49b80ac2c0 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f49b0003860 con 0x562a029d9c10 2024-09-10T23:10:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 10 client.25762 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 10 client.25762 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 20 client.25762 populate_metadata read hostname 'smithi067' 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a029f5730 0x562a029f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 4 client.25762 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a02950bf0 con 0x562a029f5730 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.773+0000 7f49c5e00640 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a029f5730 0x562a029f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.774+0000 7f49c5e00640 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a029f5730 0x562a029f7b50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05ae8d61f7025a8 server_cookie=f1cb55dc71ba7165 in_seq=0 out_seq=0 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.774+0000 7f49ab400640 10 client.25762 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49ab400640 1 -- 172.21.15.67:0/3371648736 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a02950bf0 con 0x562a029f5730 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49ab400640 10 client.25762 handle_command_reply: tid=0 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 client.25762 shutdown 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 msgr2=0x562a029e2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 0x562a029e2920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49b00126d0 tx=0x7f49b0012b90 comp rx=0 tx=0).stop 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 shutdown_connections 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a029f5730 0x562a029f7b50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029e5ba0 0x562a029e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029d9c10 0x562a029e2920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 --2- 172.21.15.67:0/3371648736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d92f0 0x562a029e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 >> 172.21.15.67:0/3371648736 conn(0x562a029d7120 msgr2=0x562a029dccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49ab400640 0 client.25762 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.775+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 shutdown_connections 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.776+0000 7f49cf49f740 1 -- 172.21.15.67:0/3371648736 wait complete. 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.776+0000 7f49cf49f740 20 client.25762 trim_cache size 0 max 16384 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 Processor -- start 2024-09-10T23:10:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 -- start start 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81b4730 con 0x562a029d92f0 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81b2000 con 0x562a029d9c10 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.798+0000 7f49cf49f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81afe70 con 0x562a029e5ba0 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51202/0 (socket says 172.21.15.67:51202) 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 -- 172.21.15.67:0/4209335520 learned_addr learned my addr 172.21.15.67:0/4209335520 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c6800640 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49bfe00640 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 -- 172.21.15.67:0/4209335520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 msgr2=0x562a029e74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 -- 172.21.15.67:0/4209335520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 msgr2=0x562a029e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.799+0000 7f49c5e00640 1 -- 172.21.15.67:0/4209335520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a029e0e20 con 0x562a029d92f0 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.800+0000 7f49c5e00640 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49b80a8d80 tx=0x7f49b80ecb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449099105020bb8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.800+0000 7f49b6600640 1 -- 172.21.15.67:0/4209335520 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80ac100 con 0x562a029d92f0 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.800+0000 7f49b6600640 1 -- 172.21.15.67:0/4209335520 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49b80ac2c0 con 0x562a029d92f0 2024-09-10T23:10:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.800+0000 7f49b6600640 1 -- 172.21.15.67:0/4209335520 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80ac580 con 0x562a029d92f0 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 -- 172.21.15.67:0/4209335520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 msgr2=0x562a029f1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49b80a8d80 tx=0x7f49b80ecb20 comp rx=0 tx=0).stop 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 -- 172.21.15.67:0/4209335520 shutdown_connections 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 --2- 172.21.15.67:0/4209335520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029f1a80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 -- 172.21.15.67:0/4209335520 >> 172.21.15.67:0/4209335520 conn(0x562a029d7120 msgr2=0x562a029e4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 -- 172.21.15.67:0/4209335520 shutdown_connections 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.801+0000 7f49cf49f740 1 -- 172.21.15.67:0/4209335520 wait complete. 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.802+0000 7f49cf49f740 1 -- start start 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.802+0000 7f49cf49f740 1 Processor -- start 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81b04c0 con 0x562a029d92f0 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81b1930 con 0x562a029d9c10 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49c81b0b90 con 0x562a029e5ba0 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49cf49f740 10 client.0 _ll_register_callbacks cb 0x7f49b0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51216/0 (socket says 172.21.15.67:51216) 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49c5e00640 1 -- 172.21.15.67:0/1084348656 learned_addr learned my addr 172.21.15.67:0/1084348656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.803+0000 7f49bfe00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 -- 172.21.15.67:0/1084348656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 msgr2=0x562a029e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 -- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 msgr2=0x562a029e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 -- 172.21.15.67:0/1084348656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a029e96f0 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49c5e00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49b80ac740 tx=0x7f49b80b10d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0377e33f8eb09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49ab400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80ac580 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9519+0+0 (secure 0 0 0) 0x7f49b80eccb0 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.804+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b80ac2c0 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.805+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f49b80abf20 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.805+0000 7f49cf49f740 10 client.16053 fetch_fsmap learned FSMap version 31 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.805+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a02a63540 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.805+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49b80ac2c0 con 0x562a029d92f0 2024-09-10T23:10:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f49b8005290 con 0x562a029d92f0 2024-09-10T23:10:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 10 client.16053 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:10:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 10 client.16053 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 20 client.16053 populate_metadata read hostname 'smithi067' 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a02a63760 0x562a02a65b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 4 client.16053 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x562a0294c6a0 con 0x562a02a63760 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.806+0000 7f49bfe00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a02a63760 0x562a02a65b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.807+0000 7f49ab400640 10 client.16053 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:10:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.807+0000 7f49bfe00640 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a02a63760 0x562a02a65b50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b7f281dc46b4b0 server_cookie=5cba1adf898d88d1 in_seq=0 out_seq=0 2024-09-10T23:10:55.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:55 smithi067 ceph-mon[30939]: pgmap v758: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:10:55.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:55.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:55.824 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:10:55.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.936+0000 7f49ab400640 1 -- 172.21.15.67:0/1084348656 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562a0294c6a0 con 0x562a02a63760 2024-09-10T23:10:55.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.936+0000 7f49ab400640 10 client.16053 handle_command_reply: tid=0 2024-09-10T23:10:55.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.936+0000 7f49cf49f740 1 client.16053 shutdown 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.936+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 msgr2=0x562a029ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.936+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49b80ac740 tx=0x7f49b80b10d0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 shutdown_connections 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562a02a63760 0x562a02a65b50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a029e5ba0 0x562a029e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a029d9c10 0x562a029e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49ab400640 0 client.16053 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 --2- 172.21.15.67:0/1084348656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562a029d92f0 0x562a029ee170 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 >> 172.21.15.67:0/1084348656 conn(0x562a029d7120 msgr2=0x562a029e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 shutdown_connections 2024-09-10T23:10:55.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 1 -- 172.21.15.67:0/1084348656 wait complete. 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.937+0000 7f49cf49f740 20 client.16053 trim_cache size 0 max 16384 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 34, 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:10:55.939 INFO:teuthology.orchestra.run.smithi067.stdout: "34f4647": { 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 34, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.4, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:10:55.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:10:55.941 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 msgr2=0x7f49ac0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 0x7f49ac0b7a40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006010 tx=0x7f49c00098c0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 msgr2=0x7f49c819b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9f0 tx=0x7f49b800edf0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 shutdown_connections 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49ac0b5580 0x7f49ac0b7a40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49c810f250 0x7f49c81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49c8101fc0 0x7f49c819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 --2- 172.21.15.67:0/2328859482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49c81016a0 0x7f49c819b0d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 >> 172.21.15.67:0/2328859482 conn(0x7f49c80fd5b0 msgr2=0x7f49c81113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.938+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 shutdown_connections 2024-09-10T23:10:55.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:10:55.939+0000 7f49c7200640 1 -- 172.21.15.67:0/2328859482 wait complete. 2024-09-10T23:10:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:55 smithi092 ceph-mon[34223]: pgmap v758: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:10:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:10:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:55 smithi145 ceph-mon[34397]: pgmap v758: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:10:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:10:56.805 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '34f4647', quiesce took 4.8 seconds. Will release after: 20.599999999999998 2024-09-10T23:10:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:57 smithi092 ceph-mon[34223]: pgmap v759: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:57 smithi067 ceph-mon[30939]: pgmap v759: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:10:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:57 smithi145 ceph-mon[34397]: pgmap v759: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:11:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:10:59 smithi092 ceph-mon[34223]: pgmap v760: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:11:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:10:59 smithi067 ceph-mon[30939]: pgmap v760: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:11:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:10:59 smithi145 ceph-mon[34397]: pgmap v760: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:11:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:01 smithi092 ceph-mon[34223]: pgmap v761: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:11:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:01 smithi067 ceph-mon[30939]: pgmap v761: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:11:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:01 smithi145 ceph-mon[34397]: pgmap v761: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:11:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:03 smithi092 ceph-mon[34223]: pgmap v762: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.4 MiB/s wr, 24 op/s 2024-09-10T23:11:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:03 smithi067 ceph-mon[30939]: pgmap v762: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.4 MiB/s wr, 24 op/s 2024-09-10T23:11:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:03 smithi145 ceph-mon[34397]: pgmap v762: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.4 MiB/s wr, 24 op/s 2024-09-10T23:11:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:05 smithi092 ceph-mon[34223]: pgmap v763: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 26 op/s 2024-09-10T23:11:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:05 smithi067 ceph-mon[30939]: pgmap v763: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 26 op/s 2024-09-10T23:11:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:05 smithi145 ceph-mon[34397]: pgmap v763: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 26 op/s 2024-09-10T23:11:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:07 smithi092 ceph-mon[34223]: pgmap v764: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.6 MiB/s wr, 26 op/s 2024-09-10T23:11:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:07 smithi067 ceph-mon[30939]: pgmap v764: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.6 MiB/s wr, 26 op/s 2024-09-10T23:11:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:07 smithi145 ceph-mon[34397]: pgmap v764: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.6 MiB/s wr, 26 op/s 2024-09-10T23:11:09.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:08 smithi092 ceph-mon[34223]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T23:11:09.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:08 smithi145 ceph-mon[34397]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T23:11:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:08 smithi067 ceph-mon[30939]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T23:11:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:09 smithi092 ceph-mon[34223]: pgmap v766: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-09-10T23:11:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:09 smithi067 ceph-mon[30939]: pgmap v766: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-09-10T23:11:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:09 smithi145 ceph-mon[34397]: pgmap v766: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-09-10T23:11:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:11 smithi092 ceph-mon[34223]: pgmap v767: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 720 KiB/s wr, 27 op/s 2024-09-10T23:11:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:11 smithi067 ceph-mon[30939]: pgmap v767: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 720 KiB/s wr, 27 op/s 2024-09-10T23:11:12.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:11 smithi145 ceph-mon[34397]: pgmap v767: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 720 KiB/s wr, 27 op/s 2024-09-10T23:11:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:13 smithi092 ceph-mon[34223]: pgmap v768: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 694 KiB/s wr, 10 op/s 2024-09-10T23:11:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:13 smithi092 ceph-mon[34223]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T23:11:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:13 smithi067 ceph-mon[30939]: pgmap v768: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 694 KiB/s wr, 10 op/s 2024-09-10T23:11:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:13 smithi067 ceph-mon[30939]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T23:11:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:13 smithi145 ceph-mon[34397]: pgmap v768: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 694 KiB/s wr, 10 op/s 2024-09-10T23:11:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:13 smithi145 ceph-mon[34397]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T23:11:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:15 smithi092 ceph-mon[34223]: pgmap v770: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-10T23:11:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:15 smithi067 ceph-mon[30939]: pgmap v770: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-10T23:11:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:15 smithi145 ceph-mon[34397]: pgmap v770: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-10T23:11:17.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:11:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:18.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:17 smithi067 ceph-mon[30939]: pgmap v771: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 357 B/s wr, 7 op/s 2024-09-10T23:11:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:17 smithi092 ceph-mon[34223]: pgmap v771: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 357 B/s wr, 7 op/s 2024-09-10T23:11:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:17 smithi145 ceph-mon[34397]: pgmap v771: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 357 B/s wr, 7 op/s 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 -- 172.21.15.67:0/2765796353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f394810b1a0 msgr2=0x7f39481119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 --2- 172.21.15.67:0/2765796353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f394810b1a0 0x7f39481119a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3940009900 tx=0x7f394002a410 comp rx=0 tx=0).stop 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 -- 172.21.15.67:0/2765796353 shutdown_connections 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 --2- 172.21.15.67:0/2765796353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f394810b1a0 0x7f39481119a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 --2- 172.21.15.67:0/2765796353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810a820 0x7f394810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 --2- 172.21.15.67:0/2765796353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3948104e10 0x7f39481051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 -- 172.21.15.67:0/2765796353 >> 172.21.15.67:0/2765796353 conn(0x7f3948100840 msgr2=0x7f3948102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.502+0000 7f3947e00640 1 -- 172.21.15.67:0/2765796353 shutdown_connections 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.503+0000 7f3947e00640 1 -- 172.21.15.67:0/2765796353 wait complete. 2024-09-10T23:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.503+0000 7f3947e00640 1 Processor -- start 2024-09-10T23:11:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 -- start start 2024-09-10T23:11:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f394810a820 0x7f394819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810b1a0 0x7f39481a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3948114d60 con 0x7f3948104e10 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3948114be0 con 0x7f394810b1a0 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3947e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3948114ee0 con 0x7f394810a820 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3946a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3946a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46364/0 (socket says 172.21.15.67:46364) 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.504+0000 7f3946a00640 1 -- 172.21.15.67:0/3985597254 learned_addr learned my addr 172.21.15.67:0/3985597254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3947400640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810b1a0 0x7f39481a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946000640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f394810a820 0x7f394819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 -- 172.21.15.67:0/3985597254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f394810a820 msgr2=0x7f394819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f394810a820 0x7f394819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 -- 172.21.15.67:0/3985597254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810b1a0 msgr2=0x7f39481a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810b1a0 0x7f39481a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 -- 172.21.15.67:0/3985597254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39481a1200 con 0x7f3948104e10 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.505+0000 7f3946a00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f393800eb90 tx=0x7f393800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.506+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938004d20 con 0x7f3948104e10 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.506+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f393800ee90 con 0x7f3948104e10 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.506+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938010690 con 0x7f3948104e10 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.506+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39481a14f0 con 0x7f3948104e10 2024-09-10T23:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.506+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a7bb0 con 0x7f3948104e10 2024-09-10T23:11:18.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.507+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39481051f0 con 0x7f3948104e10 2024-09-10T23:11:18.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.509+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3938005020 con 0x7f3948104e10 2024-09-10T23:11:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.510+0000 7f393f400640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 0x7f392c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.511+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7f39380e6a00 con 0x7f3948104e10 2024-09-10T23:11:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.511+0000 7f3946000640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 0x7f392c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.512+0000 7f3946000640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 0x7f392c0b7a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f394810a330 tx=0x7f393000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:18.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.514+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f39380ad140 con 0x7f3948104e10 2024-09-10T23:11:18.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.748+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39481a7f50 con 0x7f3948104e10 2024-09-10T23:11:18.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.749+0000 7f393f400640 1 -- 172.21.15.67:0/3985597254 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f39380aca40 con 0x7f3948104e10 2024-09-10T23:11:18.752 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:11:18.752 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:11:18.752 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:11:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 msgr2=0x7f392c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 0x7f392c0b7a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f394810a330 tx=0x7f393000b410 comp rx=0 tx=0).stop 2024-09-10T23:11:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 msgr2=0x7f394819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f393800eb90 tx=0x7f393800c6a0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 shutdown_connections 2024-09-10T23:11:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f392c0b5580 0x7f392c0b7a40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f394810b1a0 0x7f39481a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f394810a820 0x7f394819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 --2- 172.21.15.67:0/3985597254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3948104e10 0x7f394819b150 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 >> 172.21.15.67:0/3985597254 conn(0x7f3948100840 msgr2=0x7f3948102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 shutdown_connections 2024-09-10T23:11:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:18.752+0000 7f3947e00640 1 -- 172.21.15.67:0/3985597254 wait complete. 2024-09-10T23:11:19.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3985597254' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:19.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3985597254' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3985597254' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:19.575 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 34f4647 --release --await' 2024-09-10T23:11:19.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 34f4647 --release --await 2024-09-10T23:11:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:19 smithi067 ceph-mon[30939]: pgmap v772: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:11:19.957 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:19 smithi092 ceph-mon[34223]: pgmap v772: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:11:20.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:19 smithi145 ceph-mon[34397]: pgmap v772: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:11:20.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.561+0000 7ff1a7200640 1 -- 172.21.15.67:0/1753745302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a810f220 msgr2=0x7ff1a8111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.561+0000 7ff1a7200640 1 --2- 172.21.15.67:0/1753745302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a810f220 0x7ff1a8111610 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009ca0 tx=0x7ff1a002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.561+0000 7ff1a7200640 1 -- 172.21.15.67:0/1753745302 shutdown_connections 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.561+0000 7ff1a7200640 1 --2- 172.21.15.67:0/1753745302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a810f220 0x7ff1a8111610 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.562+0000 7ff1a7200640 1 --2- 172.21.15.67:0/1753745302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a8075de0 0x7ff1a810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.562+0000 7ff1a7200640 1 --2- 172.21.15.67:0/1753745302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a80754c0 0x7ff1a80758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.562+0000 7ff1a7200640 1 -- 172.21.15.67:0/1753745302 >> 172.21.15.67:0/1753745302 conn(0x7ff1a80fe6c0 msgr2=0x7ff1a8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.562+0000 7ff1a7200640 1 -- 172.21.15.67:0/1753745302 shutdown_connections 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.562+0000 7ff1a7200640 1 -- 172.21.15.67:0/1753745302 wait complete. 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 Processor -- start 2024-09-10T23:11:20.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 -- start start 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a80754c0 0x7ff1a819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a810f220 0x7ff1a81a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8114d00 con 0x7ff1a8075de0 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8114b80 con 0x7ff1a810f220 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8114e80 con 0x7ff1a80754c0 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37130/0 (socket says 172.21.15.67:37130) 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a5400640 1 -- 172.21.15.67:0/3648402674 learned_addr learned my addr 172.21.15.67:0/3648402674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.563+0000 7ff1a6800640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a810f220 0x7ff1a81a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a80754c0 0x7ff1a819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 -- 172.21.15.67:0/3648402674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a80754c0 msgr2=0x7ff1a819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a80754c0 0x7ff1a819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 -- 172.21.15.67:0/3648402674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a810f220 msgr2=0x7ff1a81a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a810f220 0x7ff1a81a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 -- 172.21.15.67:0/3648402674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a81a1260 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a5400640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9f0 tx=0x7ff19400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff19ea00640 1 -- 172.21.15.67:0/3648402674 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19400cd80 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.564+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a81a1550 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.565+0000 7ff19ea00640 1 -- 172.21.15.67:0/3648402674 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1940054b0 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.565+0000 7ff19ea00640 1 -- 172.21.15.67:0/3648402674 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194010430 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.565+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a81a7c10 con 0x7ff1a8075de0 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 Processor -- start 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 -- start start 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b296540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b298e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff19ea00640 1 -- 172.21.15.67:0/3648402674 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff194005020 con 0x7ff1a8075de0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56514b201290 con 0x56514b28a1d0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56514b1f06c0 con 0x56514b28aaf0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.568+0000 7ff1af7be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56514b1f44d0 con 0x56514b296a80 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37132/0 (socket says 172.21.15.67:37132) 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 -- 172.21.15.67:0/2031823740 learned_addr learned my addr 172.21.15.67:0/2031823740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b296540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a5400640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b298e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 -- 172.21.15.67:0/2031823740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 msgr2=0x56514b298e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b298e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 -- 172.21.15.67:0/2031823740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 msgr2=0x56514b296540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b296540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a6800640 1 -- 172.21.15.67:0/2031823740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514b17d460 con 0x56514b28a1d0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.569+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b296540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009ca0 tx=0x7ff1a0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e13f397076b6136 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff193000640 1 -- 172.21.15.67:0/2031823740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00316d0 con 0x56514b28a1d0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff193000640 1 -- 172.21.15.67:0/2031823740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a0036070 con 0x56514b28a1d0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff193000640 1 -- 172.21.15.67:0/2031823740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00042a0 con 0x56514b28a1d0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff19ea00640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 0x7ff18c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff19ea00640 1 -- 172.21.15.67:0/3648402674 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ff1940e6870 con 0x7ff1a8075de0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.570+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 0x7ff18c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 -- 172.21.15.67:0/2031823740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 msgr2=0x56514b28a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009ca0 tx=0x7ff1a0005f30 comp rx=0 tx=0).stop 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 -- 172.21.15.67:0/2031823740 shutdown_connections 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b298e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b296540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2031823740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28a1d0 0x56514b28a5b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 -- 172.21.15.67:0/2031823740 >> 172.21.15.67:0/2031823740 conn(0x56514b288000 msgr2=0x56514b2891a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 -- 172.21.15.67:0/2031823740 shutdown_connections 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1af7be740 1 -- 172.21.15.67:0/2031823740 wait complete. 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.571+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 0x7ff18c0b7a40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x56514b28d350 tx=0x7ff198005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 -- start start 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 Processor -- start 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 0x56514b293a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b293f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56514b051bb0 con 0x56514b296a80 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56514b18b6c0 con 0x56514b28aaf0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56514b299550 con 0x56514b28a1d0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.572+0000 7ff1af7be740 10 client.0 _ll_register_callbacks cb 0x56514b1a6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37138/0 (socket says 172.21.15.67:37138) 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 -- 172.21.15.67:0/2469958819 learned_addr learned my addr 172.21.15.67:0/2469958819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a5400640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b293f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 0x56514b293a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 -- 172.21.15.67:0/2469958819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 msgr2=0x56514b293a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 0x56514b293a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 -- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 msgr2=0x56514b293f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b293f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 -- 172.21.15.67:0/2469958819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514af4eb00 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 0x56514b293a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009bf0 tx=0x7ff1a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cac93b9997b40d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.573+0000 7ff187400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0051020 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ff1a0031d90 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a0036070 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1a00305c0 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff1af7be740 10 client.16071 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56514afc5960 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.574+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a0036070 con 0x56514b296a80 2024-09-10T23:11:20.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff1a0043020 con 0x56514b296a80 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 10 client.16071 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 20 client.16071 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b2a62d0 0x56514b2a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 4 client.16071 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514b2031e0 con 0x56514b2a62d0 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.575+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b2a62d0 0x56514b2a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.576+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b2a62d0 0x56514b2a86c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a86458f6068e6d7 server_cookie=71136e95a13633f2 in_seq=0 out_seq=0 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.576+0000 7ff187400640 10 client.16071 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff187400640 1 -- 172.21.15.67:0/2469958819 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56514b2031e0 con 0x56514b2a62d0 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff187400640 10 client.16071 handle_command_reply: tid=0 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 client.16071 shutdown 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 msgr2=0x56514b2944b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009bf0 tx=0x7ff1a0003c20 comp rx=0 tx=0).stop 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 shutdown_connections 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b2a62d0 0x56514b2a86c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b2944b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28aaf0 0x56514b293f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2469958819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28a1d0 0x56514b293a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff187400640 0 client.16071 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 >> 172.21.15.67:0/2469958819 conn(0x56514b288000 msgr2=0x56514b28dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 shutdown_connections 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 1 -- 172.21.15.67:0/2469958819 wait complete. 2024-09-10T23:11:20.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.577+0000 7ff1af7be740 20 client.16071 trim_cache size 0 max 16384 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 Processor -- start 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 -- start start 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b28da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 0x56514b290c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81b4650 con 0x56514b296a80 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81b1f20 con 0x56514b28a1d0 2024-09-10T23:11:20.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.596+0000 7ff1af7be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81afd90 con 0x56514b28aaf0 2024-09-10T23:11:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37146/0 (socket says 172.21.15.67:37146) 2024-09-10T23:11:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 -- 172.21.15.67:0/3614012133 learned_addr learned my addr 172.21.15.67:0/3614012133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b28da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a5400640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 0x56514b290c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 -- 172.21.15.67:0/3614012133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 msgr2=0x56514b290c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 0x56514b290c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 -- 172.21.15.67:0/3614012133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 msgr2=0x56514b28da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b28da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a6800640 1 -- 172.21.15.67:0/3614012133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514b2995a0 con 0x56514b296a80 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b28da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.597+0000 7ff1a5400640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 0x56514b290c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff1a6800640 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff1a002ade0 tx=0x7ff1a00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f46841adffdcb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff192600640 1 -- 172.21.15.67:0/3614012133 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a005f070 con 0x56514b296a80 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff192600640 1 -- 172.21.15.67:0/3614012133 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a0036040 con 0x56514b296a80 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff192600640 1 -- 172.21.15.67:0/3614012133 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0051040 con 0x56514b296a80 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff1af7be740 1 -- 172.21.15.67:0/3614012133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 msgr2=0x56514b291180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.598+0000 7ff1af7be740 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff1a002ade0 tx=0x7ff1a00046a0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 -- 172.21.15.67:0/3614012133 shutdown_connections 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b296a80 0x56514b291180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b28aaf0 0x56514b290c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 --2- 172.21.15.67:0/3614012133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b28da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 -- 172.21.15.67:0/3614012133 >> 172.21.15.67:0/3614012133 conn(0x56514b288000 msgr2=0x56514b28d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 -- 172.21.15.67:0/3614012133 shutdown_connections 2024-09-10T23:11:20.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 -- 172.21.15.67:0/3614012133 wait complete. 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 -- start start 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.599+0000 7ff1af7be740 1 Processor -- start 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b29b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b29c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81b03e0 con 0x56514b28aaf0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81b1850 con 0x56514b28a1d0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81b0ab0 con 0x56514b296a80 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1af7be740 10 client.0 _ll_register_callbacks cb 0x7ff198009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37162/0 (socket says 172.21.15.67:37162) 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1a5400640 1 -- 172.21.15.67:0/2324735068 learned_addr learned my addr 172.21.15.67:0/2324735068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b29b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.600+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b29c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 -- 172.21.15.67:0/2324735068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 msgr2=0x56514b29c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b29c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 -- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 msgr2=0x56514b29b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b29b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 -- 172.21.15.67:0/2324735068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514b299260 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a6800640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b29c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b29b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1a5400640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ac840 tx=0x7ff1940053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cd70df7a25a721 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff187400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1940ac680 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ff1940adde0 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a0036040 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.601+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1940ac020 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 10 client.16077 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56514b288a80 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a0036040 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff1940b59b0 con 0x56514b28aaf0 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 10 client.16077 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 10 client.16077 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 20 client.16077 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b314680 0x56514b316aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 4 client.16077 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "34f4647", "release": true, "await": true}] 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.602+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "34f4647", "release": true, "await": true}) -- 0x56514b28f600 con 0x56514b314680 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.603+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b314680 0x56514b316aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.603+0000 7ff1a5e00640 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b314680 0x56514b316aa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd92e2cd96881b42 server_cookie=8bdfb43fe7e93585 in_seq=0 out_seq=0 2024-09-10T23:11:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.603+0000 7ff187400640 10 client.16077 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.628+0000 7ff187400640 1 -- 172.21.15.67:0/2324735068 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56514b28f600 con 0x56514b314680 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.628+0000 7ff187400640 10 client.16077 handle_command_reply: tid=0 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.628+0000 7ff1af7be740 1 client.16077 shutdown 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 msgr2=0x56514b29bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ac840 tx=0x7ff1940053b0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 shutdown_connections 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56514b314680 0x56514b316aa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56514b296a80 0x56514b29c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56514b28aaf0 0x56514b29bcb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 --2- 172.21.15.67:0/2324735068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56514b28a1d0 0x56514b29b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 >> 172.21.15.67:0/2324735068 conn(0x56514b288000 msgr2=0x56514b28dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff187400640 0 client.16077 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 shutdown_connections 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 1 -- 172.21.15.67:0/2324735068 wait complete. 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.629+0000 7ff1af7be740 20 client.16077 trim_cache size 0 max 16384 2024-09-10T23:11:20.632 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 36, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "34f4647": { 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 36, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.4, 2024-09-10T23:11:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:20.634 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 msgr2=0x7ff18c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 0x7ff18c0b7a40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x56514b28d350 tx=0x7ff198005f90 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 msgr2=0x7ff1a819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9f0 tx=0x7ff19400edf0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 shutdown_connections 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff18c0b5580 0x7ff18c0b7a40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1a810f220 0x7ff1a81a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1a8075de0 0x7ff1a819b6f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 --2- 172.21.15.67:0/3648402674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1a80754c0 0x7ff1a819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 >> 172.21.15.67:0/3648402674 conn(0x7ff1a80fe6c0 msgr2=0x7ff1a80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.630+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 shutdown_connections 2024-09-10T23:11:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:20.631+0000 7ff1a7200640 1 -- 172.21.15.67:0/3648402674 wait complete. 2024-09-10T23:11:21.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '34f4647', total seconds elapsed: 29.6 2024-09-10T23:11:21.666 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-10T23:11:21.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:11:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:21 smithi092 ceph-mon[34223]: pgmap v773: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:21 smithi067 ceph-mon[30939]: pgmap v773: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:22.104 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:21 smithi145 ceph-mon[34397]: pgmap v773: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 -- 172.21.15.67:0/1067627868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f256810a9b0 msgr2=0x7f256810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 --2- 172.21.15.67:0/1067627868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f256810a9b0 0x7f256810adf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f255402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 -- 172.21.15.67:0/1067627868 shutdown_connections 2024-09-10T23:11:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 --2- 172.21.15.67:0/1067627868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810b330 0x7f2568111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 --2- 172.21.15.67:0/1067627868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f256810a9b0 0x7f256810adf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 --2- 172.21.15.67:0/1067627868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2568104fa0 0x7f2568105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 -- 172.21.15.67:0/1067627868 >> 172.21.15.67:0/1067627868 conn(0x7f2568100a50 msgr2=0x7f2568102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:22.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 -- 172.21.15.67:0/1067627868 shutdown_connections 2024-09-10T23:11:22.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.813+0000 7f256da00640 1 -- 172.21.15.67:0/1067627868 wait complete. 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.814+0000 7f256da00640 1 Processor -- start 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 -- start start 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 0x7f25681105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 0x7f2568110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256810b330 0x7f25681ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2568078640 con 0x7f2568104fa0 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f25680784c0 con 0x7f256810a9b0 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f256da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f25680787c0 con 0x7f256810b330 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f2566a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 0x7f25681105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:22.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f2566a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 0x7f25681105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37172/0 (socket says 172.21.15.67:37172) 2024-09-10T23:11:22.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f2566000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 0x7f2568110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:22.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.815+0000 7f2566a00640 1 -- 172.21.15.67:0/1874490937 learned_addr learned my addr 172.21.15.67:0/1874490937 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2567400640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256810b330 0x7f25681ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 -- 172.21.15.67:0/1874490937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256810b330 msgr2=0x7f25681ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256810b330 0x7f25681ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 -- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 msgr2=0x7f25681105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 0x7f25681105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 -- 172.21.15.67:0/1874490937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681aba30 con 0x7f256810a9b0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.816+0000 7f2566000640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 0x7f2568110b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f255402a910 tx=0x7f2554003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.817+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255403f070 con 0x7f256810a9b0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.817+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2554003e10 con 0x7f256810a9b0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.817+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25681abcc0 con 0x7f256810a9b0 2024-09-10T23:11:22.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.817+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554035030 con 0x7f256810a9b0 2024-09-10T23:11:22.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.817+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681ac1a0 con 0x7f256810a9b0 2024-09-10T23:11:22.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.818+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2568105380 con 0x7f256810a9b0 2024-09-10T23:11:22.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.821+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2554031400 con 0x7f256810a9b0 2024-09-10T23:11:22.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.822+0000 7f255f400640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 0x7f254c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:22.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.823+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7f2554107520 con 0x7f256810a9b0 2024-09-10T23:11:22.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.823+0000 7f2566a00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 0x7f254c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:22.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.823+0000 7f2566a00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 0x7f254c0b7910 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2558009a40 tx=0x7f25580091c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:22.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:22.824+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f25540cdc60 con 0x7f256810a9b0 2024-09-10T23:11:23.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.135+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f256807a510 con 0x7f256810a9b0 2024-09-10T23:11:23.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.136+0000 7f255f400640 1 -- 172.21.15.67:0/1874490937 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f25540cd560 con 0x7f256810a9b0 2024-09-10T23:11:23.138 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:11:23.138 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:11:23.138 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:11:23.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.139+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 msgr2=0x7f254c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:23.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 0x7f254c0b7910 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2558009a40 tx=0x7f25580091c0 comp rx=0 tx=0).stop 2024-09-10T23:11:23.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 msgr2=0x7f2568110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:23.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 0x7f2568110b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f255402a910 tx=0x7f2554003c20 comp rx=0 tx=0).stop 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 shutdown_connections 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f254c0b5450 0x7f254c0b7910 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256810b330 0x7f25681ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f256810a9b0 0x7f2568110b10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 --2- 172.21.15.67:0/1874490937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2568104fa0 0x7f25681105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 >> 172.21.15.67:0/1874490937 conn(0x7f2568100a50 msgr2=0x7f2568102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 shutdown_connections 2024-09-10T23:11:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:23.140+0000 7f256da00640 1 -- 172.21.15.67:0/1874490937 wait complete. 2024-09-10T23:11:24.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:23 smithi067 ceph-mon[30939]: pgmap v774: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:24.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:23 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1874490937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:24.064 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-10T23:11:24.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-10T23:11:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:23 smithi145 ceph-mon[34397]: pgmap v774: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:23 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1874490937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:23 smithi092 ceph-mon[34223]: pgmap v774: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:11:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:23 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1874490937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:24.514 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 -- 172.21.15.67:0/3721169411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108fa0 msgr2=0x7ffb2810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3721169411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108fa0 0x7ffb2810f7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009ca0 tx=0x7ffb2002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 -- 172.21.15.67:0/3721169411 shutdown_connections 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3721169411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108fa0 0x7ffb2810f7a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3721169411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108620 0x7ffb28108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3721169411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 -- 172.21.15.67:0/3721169411 >> 172.21.15.67:0/3721169411 conn(0x7ffb280fe6c0 msgr2=0x7ffb28100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 -- 172.21.15.67:0/3721169411 shutdown_connections 2024-09-10T23:11:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.274+0000 7ffb27e00640 1 -- 172.21.15.67:0/3721169411 wait complete. 2024-09-10T23:11:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 Processor -- start 2024-09-10T23:11:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 -- start start 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 0x7ffb28199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108fa0 0x7ffb2819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28112b20 con 0x7ffb28108620 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281129a0 con 0x7ffb28102c10 2024-09-10T23:11:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.276+0000 7ffb27e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28112ca0 con 0x7ffb28108fa0 2024-09-10T23:11:25.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:55064/0 (socket says 172.21.15.67:55064) 2024-09-10T23:11:25.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 -- 172.21.15.67:0/3165693811 learned_addr learned my addr 172.21.15.67:0/3165693811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26000640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 0x7ffb28199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb27400640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108fa0 0x7ffb2819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 -- 172.21.15.67:0/3165693811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108fa0 msgr2=0x7ffb2819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108fa0 0x7ffb2819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.277+0000 7ffb26a00640 1 -- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 msgr2=0x7ffb28199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb26a00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 0x7ffb28199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb26a00640 1 -- 172.21.15.67:0/3165693811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2819f000 con 0x7ffb28102c10 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb26000640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 0x7ffb28199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb26a00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9f0 tx=0x7ffb1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb1f400640 1 -- 172.21.15.67:0/3165693811 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800cd80 con 0x7ffb28102c10 2024-09-10T23:11:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb1f400640 1 -- 172.21.15.67:0/3165693811 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb180108c0 con 0x7ffb28102c10 2024-09-10T23:11:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.278+0000 7ffb1f400640 1 -- 172.21.15.67:0/3165693811 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180187c0 con 0x7ffb28102c10 2024-09-10T23:11:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.279+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2819f2f0 con 0x7ffb28102c10 2024-09-10T23:11:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.279+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb281a59b0 con 0x7ffb28102c10 2024-09-10T23:11:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.283+0000 7ffb1f400640 1 -- 172.21.15.67:0/3165693811 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ffb18010430 con 0x7ffb28102c10 2024-09-10T23:11:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.283+0000 7ffb3068b740 1 Processor -- start 2024-09-10T23:11:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 -- start start 2024-09-10T23:11:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b288cacca0 con 0x55b288d372f0 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b288c9c0d0 con 0x55b288d43ba0 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb3068b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b288c9fee0 con 0x55b288d37c10 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.284+0000 7ffb27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37202/0 (socket says 172.21.15.67:37202) 2024-09-10T23:11:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 -- 172.21.15.67:0/218746466 learned_addr learned my addr 172.21.15.67:0/218746466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb1f400640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 0x7ffb0c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb26000640 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb1f400640 1 -- 172.21.15.67:0/3165693811 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ffb180e6770 con 0x7ffb28102c10 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 -- 172.21.15.67:0/218746466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 msgr2=0x55b288d43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 -- 172.21.15.67:0/218746466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 msgr2=0x55b288d45f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d45f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.285+0000 7ffb27400640 1 -- 172.21.15.67:0/218746466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b288c4e400 con 0x55b288d372f0 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.286+0000 7ffb26a00640 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.286+0000 7ffb27400640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 0x7ffb0c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.286+0000 7ffb27400640 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb20043c60 tx=0x7ffb20043c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e43504e91ba515 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.286+0000 7ffb13e00640 1 -- 172.21.15.67:0/218746466 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20049030 con 0x55b288d372f0 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.287+0000 7ffb13e00640 1 -- 172.21.15.67:0/218746466 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb2004b070 con 0x55b288d372f0 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.287+0000 7ffb13e00640 1 -- 172.21.15.67:0/218746466 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20004040 con 0x55b288d372f0 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 -- 172.21.15.67:0/218746466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 msgr2=0x55b288d376d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb27400640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 0x7ffb0c0b7a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb20052000 tx=0x7ffb20053000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb20043c60 tx=0x7ffb20043c90 comp rx=0 tx=0).stop 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 -- 172.21.15.67:0/218746466 shutdown_connections 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 --2- 172.21.15.67:0/218746466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d376d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 -- 172.21.15.67:0/218746466 >> 172.21.15.67:0/218746466 conn(0x55b288d35120 msgr2=0x55b288d362c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 -- 172.21.15.67:0/218746466 shutdown_connections 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.288+0000 7ffb3068b740 1 -- 172.21.15.67:0/218746466 wait complete. 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.289+0000 7ffb3068b740 1 -- start start 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.289+0000 7ffb3068b740 1 Processor -- start 2024-09-10T23:11:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b288afd7a0 con 0x55b288d372f0 2024-09-10T23:11:25.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b288c41a10 con 0x55b288d43ba0 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b288d46820 con 0x55b288d37c10 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.290+0000 7ffb3068b740 10 client.0 _ll_register_callbacks cb 0x55b288c51be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37204/0 (socket says 172.21.15.67:37204) 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb26a00640 1 -- 172.21.15.67:0/258786774 learned_addr learned my addr 172.21.15.67:0/258786774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb26000640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb27400640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb27400640 1 -- 172.21.15.67:0/258786774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 msgr2=0x55b288d40850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.291+0000 7ffb27400640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb27400640 1 -- 172.21.15.67:0/258786774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 msgr2=0x55b288d43400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb27400640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb27400640 1 -- 172.21.15.67:0/258786774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b288a09b90 con 0x55b288d43ba0 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb26a00640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb27400640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d40dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009ca0 tx=0x7ffb20004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b76cc9a130a2b686 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.292+0000 7ffb10c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:11:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20067060 con 0x55b288d43ba0 2024-09-10T23:11:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb2004b070 con 0x55b288d43ba0 2024-09-10T23:11:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ffb20009800 con 0x55b288d43ba0 2024-09-10T23:11:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb2006e050 con 0x55b288d43ba0 2024-09-10T23:11:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb3068b740 10 client.35807 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b288d52100 con 0x55b288d43ba0 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.293+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb2004b070 con 0x55b288d43ba0 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffb200305c0 con 0x55b288d43ba0 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 10 client.35807 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 10 client.35807 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 20 client.35807 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288d53450 0x55b288d55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 4 client.35807 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.294+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b288caebf0 con 0x55b288d53450 2024-09-10T23:11:25.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.295+0000 7ffb26a00640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288d53450 0x55b288d55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.295+0000 7ffb26a00640 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288d53450 0x55b288d55840 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a40ec7c4fc86e12 server_cookie=73ae251d0d565049 in_seq=0 out_seq=0 2024-09-10T23:11:25.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.295+0000 7ffb10c00640 10 client.35807 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:25.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.296+0000 7ffb10c00640 1 -- 172.21.15.67:0/258786774 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b288caebf0 con 0x55b288d53450 2024-09-10T23:11:25.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.296+0000 7ffb10c00640 10 client.35807 handle_command_reply: tid=0 2024-09-10T23:11:25.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.296+0000 7ffb3068b740 1 client.35807 shutdown 2024-09-10T23:11:25.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.296+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 msgr2=0x55b288d40dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.296+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d40dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009ca0 tx=0x7ffb20004d70 comp rx=0 tx=0).stop 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 shutdown_connections 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288d53450 0x55b288d55840 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb10c00640 0 client.35807 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d43ba0 0x55b288d40dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 --2- 172.21.15.67:0/258786774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d372f0 0x55b288d43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 >> 172.21.15.67:0/258786774 conn(0x55b288d35120 msgr2=0x55b288d3ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 shutdown_connections 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 1 -- 172.21.15.67:0/258786774 wait complete. 2024-09-10T23:11:25.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.297+0000 7ffb3068b740 20 client.35807 trim_cache size 0 max 16384 2024-09-10T23:11:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 Processor -- start 2024-09-10T23:11:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 -- start start 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 0x55b288d3daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281b0420 con 0x55b288d43ba0 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281adcf0 con 0x55b288d372f0 2024-09-10T23:11:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb3068b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281abb60 con 0x55b288d37c10 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.320+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 0x55b288d3daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:55098/0 (socket says 172.21.15.67:55098) 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 -- 172.21.15.67:0/508680052 learned_addr learned my addr 172.21.15.67:0/508680052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 -- 172.21.15.67:0/508680052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 msgr2=0x55b288d40100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 -- 172.21.15.67:0/508680052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 msgr2=0x55b288d3daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 0x55b288d3daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 -- 172.21.15.67:0/508680052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b288d37a40 con 0x55b288d372f0 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb27400640 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 0x55b288d3daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.321+0000 7ffb26a00640 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb180a8d30 tx=0x7ffb180e8bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8cf4dcc20df90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.322+0000 7ffb13400640 1 -- 172.21.15.67:0/508680052 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180ac2c0 con 0x55b288d372f0 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.322+0000 7ffb13400640 1 -- 172.21.15.67:0/508680052 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb180aceb0 con 0x55b288d372f0 2024-09-10T23:11:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.322+0000 7ffb13400640 1 -- 172.21.15.67:0/508680052 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180e3d00 con 0x55b288d372f0 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 -- 172.21.15.67:0/508680052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 msgr2=0x55b288d36470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb180a8d30 tx=0x7ffb180e8bc0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 -- 172.21.15.67:0/508680052 shutdown_connections 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d43ba0 0x55b288d3daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d37c10 0x55b288d40100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 --2- 172.21.15.67:0/508680052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d36470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 -- 172.21.15.67:0/508680052 >> 172.21.15.67:0/508680052 conn(0x55b288d35120 msgr2=0x55b288d426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 -- 172.21.15.67:0/508680052 shutdown_connections 2024-09-10T23:11:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.323+0000 7ffb3068b740 1 -- 172.21.15.67:0/508680052 wait complete. 2024-09-10T23:11:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 -- start start 2024-09-10T23:11:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 Processor -- start 2024-09-10T23:11:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d57080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d43ba0 0x55b288dbefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281ac1b0 con 0x55b288d37c10 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281ad620 con 0x55b288d372f0 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281ac880 con 0x55b288d43ba0 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.324+0000 7ffb3068b740 10 client.0 _ll_register_callbacks cb 0x7ffb2005b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37230/0 (socket says 172.21.15.67:37230) 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 -- 172.21.15.67:0/63550567 learned_addr learned my addr 172.21.15.67:0/63550567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26a00640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb27400640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d43ba0 0x55b288dbefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 -- 172.21.15.67:0/63550567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d43ba0 msgr2=0x55b288dbefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d43ba0 0x55b288dbefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 -- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 msgr2=0x55b288d57080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d57080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.325+0000 7ffb26000640 1 -- 172.21.15.67:0/63550567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b288dbf660 con 0x55b288d37c10 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.326+0000 7ffb26000640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb14012cb0 tx=0x7ffb1400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae204070d64f3377 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.326+0000 7ffb10c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:11:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.326+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14023060 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.326+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7ffb14004030 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.326+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb180aceb0 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb1402a050 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb3068b740 10 client.16101 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b288dc09d0 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb180aceb0 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffb14003b90 con 0x55b288d37c10 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb3068b740 10 client.16101 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.327+0000 7ffb3068b740 10 client.16101 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.328+0000 7ffb3068b740 20 client.16101 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.328+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288dc1220 0x55b288dc3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.328+0000 7ffb3068b740 4 client.16101 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-10T23:11:25.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.328+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x55b288caa6a0 con 0x55b288dc1220 2024-09-10T23:11:25.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.328+0000 7ffb26a00640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288dc1220 0x55b288dc3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:25.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.329+0000 7ffb26a00640 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288dc1220 0x55b288dc3640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32ac6b04e0cd78e server_cookie=c07bbcb4f5583f9a in_seq=0 out_seq=0 2024-09-10T23:11:25.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.329+0000 7ffb10c00640 10 client.16101 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb10c00640 1 -- 172.21.15.67:0/63550567 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b288caa6a0 con 0x55b288dc1220 2024-09-10T23:11:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb10c00640 10 client.16101 handle_command_reply: tid=0 2024-09-10T23:11:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 client.16101 shutdown 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 msgr2=0x55b288d575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb14012cb0 tx=0x7ffb1400f790 comp rx=0 tx=0).stop 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 shutdown_connections 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55b288dc1220 0x55b288dc3640 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb10c00640 0 client.16101 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b288d43ba0 0x55b288dbefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b288d37c10 0x55b288d575c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 --2- 172.21.15.67:0/63550567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b288d372f0 0x55b288d57080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.431+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 >> 172.21.15.67:0/63550567 conn(0x55b288d35120 msgr2=0x55b288d3f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.432+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 shutdown_connections 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.432+0000 7ffb3068b740 1 -- 172.21.15.67:0/63550567 wait complete. 2024-09-10T23:11:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.432+0000 7ffb3068b740 20 client.16101 trim_cache size 0 max 16384 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 38, 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "1d30eef5": { 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 38, 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:11:25.435 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.4, 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:25.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:25.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:25.437 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:11:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 msgr2=0x7ffb0c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 0x7ffb0c0b7a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb20052000 tx=0x7ffb20053000 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 msgr2=0x7ffb28198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9f0 tx=0x7ffb1800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 shutdown_connections 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ffb0c0b5580 0x7ffb0c0b7a40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28108fa0 0x7ffb2819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb28108620 0x7ffb28199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 --2- 172.21.15.67:0/3165693811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb28102c10 0x7ffb28198f50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 >> 172.21.15.67:0/3165693811 conn(0x7ffb280fe6c0 msgr2=0x7ffb28100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 shutdown_connections 2024-09-10T23:11:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:25.433+0000 7ffb27e00640 1 -- 172.21.15.67:0/3165693811 wait complete. 2024-09-10T23:11:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:25 smithi067 ceph-mon[30939]: pgmap v775: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-09-10T23:11:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:25 smithi145 ceph-mon[34397]: pgmap v775: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-09-10T23:11:26.258 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d30eef5', quiesce took 4.6 seconds. Will release after: 20.799999999999997 2024-09-10T23:11:26.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:25 smithi092 ceph-mon[34223]: pgmap v775: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-09-10T23:11:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:27 smithi067 ceph-mon[30939]: pgmap v776: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 MiB/s wr, 9 op/s 2024-09-10T23:11:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:27 smithi145 ceph-mon[34397]: pgmap v776: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 MiB/s wr, 9 op/s 2024-09-10T23:11:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:27 smithi092 ceph-mon[34223]: pgmap v776: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 MiB/s wr, 9 op/s 2024-09-10T23:11:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:29 smithi067 ceph-mon[30939]: pgmap v777: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-09-10T23:11:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:29 smithi145 ceph-mon[34397]: pgmap v777: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-09-10T23:11:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:29 smithi092 ceph-mon[34223]: pgmap v777: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-09-10T23:11:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:31 smithi067 ceph-mon[30939]: pgmap v778: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:11:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:31 smithi145 ceph-mon[34397]: pgmap v778: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:11:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:31 smithi092 ceph-mon[34223]: pgmap v778: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:11:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:33 smithi067 ceph-mon[30939]: pgmap v779: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:33 smithi145 ceph-mon[34397]: pgmap v779: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:33 smithi092 ceph-mon[34223]: pgmap v779: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:35 smithi067 ceph-mon[30939]: pgmap v780: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:35 smithi145 ceph-mon[34397]: pgmap v780: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:35 smithi092 ceph-mon[34223]: pgmap v780: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-09-10T23:11:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:37 smithi145 ceph-mon[34397]: pgmap v781: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 9 op/s 2024-09-10T23:11:38.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:37 smithi092 ceph-mon[34223]: pgmap v781: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 9 op/s 2024-09-10T23:11:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:37 smithi067 ceph-mon[30939]: pgmap v781: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 9 op/s 2024-09-10T23:11:40.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:39 smithi067 ceph-mon[30939]: pgmap v782: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-09-10T23:11:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:39 smithi145 ceph-mon[34397]: pgmap v782: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-09-10T23:11:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:39 smithi092 ceph-mon[34223]: pgmap v782: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-09-10T23:11:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:41 smithi067 ceph-mon[30939]: pgmap v783: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-10T23:11:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:41 smithi145 ceph-mon[34397]: pgmap v783: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-10T23:11:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:41 smithi092 ceph-mon[34223]: pgmap v783: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-10T23:11:44.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:43 smithi145 ceph-mon[34397]: pgmap v784: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 415 KiB/s wr, 6 op/s 2024-09-10T23:11:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:43 smithi092 ceph-mon[34223]: pgmap v784: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 415 KiB/s wr, 6 op/s 2024-09-10T23:11:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:43 smithi067 ceph-mon[30939]: pgmap v784: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 415 KiB/s wr, 6 op/s 2024-09-10T23:11:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:45 smithi145 ceph-mon[34397]: pgmap v785: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 415 KiB/s wr, 5 op/s 2024-09-10T23:11:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:45 smithi092 ceph-mon[34223]: pgmap v785: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 415 KiB/s wr, 5 op/s 2024-09-10T23:11:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:45 smithi067 ceph-mon[30939]: pgmap v785: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 415 KiB/s wr, 5 op/s 2024-09-10T23:11:47.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:11:47.454 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:47.932 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:47 smithi067 ceph-mon[30939]: pgmap v786: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 415 KiB/s wr, 7 op/s 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 -- 172.21.15.67:0/3831361663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c010b180 msgr2=0x7fc3c0111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 --2- 172.21.15.67:0/3831361663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c010b180 0x7fc3c0111980 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009ca0 tx=0x7fc3b802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 -- 172.21.15.67:0/3831361663 shutdown_connections 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 --2- 172.21.15.67:0/3831361663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c010b180 0x7fc3c0111980 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 --2- 172.21.15.67:0/3831361663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 0x7fc3c010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 --2- 172.21.15.67:0/3831361663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c0104df0 0x7fc3c01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 -- 172.21.15.67:0/3831361663 >> 172.21.15.67:0/3831361663 conn(0x7fc3c0100840 msgr2=0x7fc3c0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 -- 172.21.15.67:0/3831361663 shutdown_connections 2024-09-10T23:11:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.046+0000 7fc3bf200640 1 -- 172.21.15.67:0/3831361663 wait complete. 2024-09-10T23:11:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.047+0000 7fc3bf200640 1 Processor -- start 2024-09-10T23:11:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.047+0000 7fc3bf200640 1 -- start start 2024-09-10T23:11:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 0x7fc3c019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c010b180 0x7fc3c01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0114d40 con 0x7fc3c0104df0 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0114bc0 con 0x7fc3c010a800 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bf200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0114ec0 con 0x7fc3c010b180 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35528/0 (socket says 172.21.15.67:35528) 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bde00640 1 -- 172.21.15.67:0/1609645200 learned_addr learned my addr 172.21.15.67:0/1609645200 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3bd400640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 0x7fc3c019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.048+0000 7fc3be800640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c010b180 0x7fc3c01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 -- 172.21.15.67:0/1609645200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c010b180 msgr2=0x7fc3c01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:48.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c010b180 0x7fc3c01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 -- 172.21.15.67:0/1609645200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 msgr2=0x7fc3c019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 0x7fc3c019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 -- 172.21.15.67:0/1609645200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c01a1140 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bde00640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017030 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3b0004020 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c01a1430 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.049+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00055d0 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.050+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01a7cc0 con 0x7fc3c0104df0 2024-09-10T23:11:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.051+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c01051d0 con 0x7fc3c0104df0 2024-09-10T23:11:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.053+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc3b0009a30 con 0x7fc3c0104df0 2024-09-10T23:11:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.054+0000 7fc3b6a00640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 0x7fc3a40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.055+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7fc3b00e6470 con 0x7fc3c0104df0 2024-09-10T23:11:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.055+0000 7fc3bd400640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 0x7fc3a40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.055+0000 7fc3bd400640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 0x7fc3a40b7a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0102f00 tx=0x7fc3ac00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:48.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.056+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc3b00acc30 con 0x7fc3c0104df0 2024-09-10T23:11:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:47 smithi145 ceph-mon[34397]: pgmap v786: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 415 KiB/s wr, 7 op/s 2024-09-10T23:11:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:47 smithi092 ceph-mon[34223]: pgmap v786: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 415 KiB/s wr, 7 op/s 2024-09-10T23:11:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.288+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3c019c1c0 con 0x7fc3c0104df0 2024-09-10T23:11:48.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.289+0000 7fc3b6a00640 1 -- 172.21.15.67:0/1609645200 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fc3b00ac530 con 0x7fc3c0104df0 2024-09-10T23:11:48.290 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:11:48.290 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:11:48.290 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:11:48.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.291+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 msgr2=0x7fc3a40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:48.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.291+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 0x7fc3a40b7a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0102f00 tx=0x7fc3ac00b390 comp rx=0 tx=0).stop 2024-09-10T23:11:48.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.291+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 msgr2=0x7fc3c019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.291+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).stop 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 shutdown_connections 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc3a40b5580 0x7fc3a40b7a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3c010b180 0x7fc3c01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3c010a800 0x7fc3c019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 --2- 172.21.15.67:0/1609645200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3c0104df0 0x7fc3c019b090 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 >> 172.21.15.67:0/1609645200 conn(0x7fc3c0100840 msgr2=0x7fc3c0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 shutdown_connections 2024-09-10T23:11:48.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:48.292+0000 7fc3bf200640 1 -- 172.21.15.67:0/1609645200 wait complete. 2024-09-10T23:11:49.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:48 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1609645200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:49.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:48 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1609645200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:49.172 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:48 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1609645200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:49.191 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 1d30eef5 --release --await' 2024-09-10T23:11:49.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 1d30eef5 --release --await 2024-09-10T23:11:49.571 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:50.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:49 smithi067 ceph-mon[30939]: pgmap v787: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:11:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:49 smithi145 ceph-mon[34397]: pgmap v787: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.162+0000 7feb21600640 1 -- 172.21.15.67:0/1780637973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c0776d0 msgr2=0x7feb1c077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.162+0000 7feb21600640 1 --2- 172.21.15.67:0/1780637973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c0776d0 0x7feb1c077ab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009900 tx=0x7feb0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 -- 172.21.15.67:0/1780637973 shutdown_connections 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 --2- 172.21.15.67:0/1780637973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c113990 0x7feb1c115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 --2- 172.21.15.67:0/1780637973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 --2- 172.21.15.67:0/1780637973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c0776d0 0x7feb1c077ab0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 -- 172.21.15.67:0/1780637973 >> 172.21.15.67:0/1780637973 conn(0x7feb1c100a50 msgr2=0x7feb1c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 -- 172.21.15.67:0/1780637973 shutdown_connections 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.163+0000 7feb21600640 1 -- 172.21.15.67:0/1780637973 wait complete. 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 Processor -- start 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 -- start start 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 0x7feb1c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c119240 con 0x7feb1c113990 2024-09-10T23:11:50.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1190c0 con 0x7feb1c077ff0 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb21600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1193c0 con 0x7feb1c0776d0 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb1b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb1b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53922/0 (socket says 172.21.15.67:53922) 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.164+0000 7feb1b400640 1 -- 172.21.15.67:0/3709591599 learned_addr learned my addr 172.21.15.67:0/3709591599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb12000640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1aa00640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 0x7feb1c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 -- 172.21.15.67:0/3709591599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 msgr2=0x7feb1c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 0x7feb1c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 -- 172.21.15.67:0/3709591599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 msgr2=0x7feb1c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 -- 172.21.15.67:0/3709591599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c1a5860 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1aa00640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 0x7feb1c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb12000640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb1b400640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7feb1400d720 tx=0x7feb1400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb13e00640 1 -- 172.21.15.67:0/3709591599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017070 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb13e00640 1 -- 172.21.15.67:0/3709591599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb1400de30 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb13e00640 1 -- 172.21.15.67:0/3709591599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14004c60 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.165+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c1a5b50 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.166+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1ac210 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.168+0000 7feb241f1740 1 Processor -- start 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb13e00640 1 -- 172.21.15.67:0/3709591599 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7feb14004e00 con 0x7feb1c113990 2024-09-10T23:11:50.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 -- start start 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f429910 0x55d32f435360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f4358a0 0x55d32f437c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d32f3a0290 con 0x55d32f428ff0 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d32f38f6c0 con 0x55d32f429910 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.169+0000 7feb241f1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d32f3934d0 con 0x55d32f4358a0 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53928/0 (socket says 172.21.15.67:53928) 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 -- 172.21.15.67:0/614322974 learned_addr learned my addr 172.21.15.67:0/614322974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb1aa00640 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f429910 0x55d32f435360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 -- 172.21.15.67:0/614322974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f4358a0 msgr2=0x55d32f437c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f4358a0 0x55d32f437c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 -- 172.21.15.67:0/614322974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f429910 msgr2=0x55d32f435360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f429910 0x55d32f435360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 -- 172.21.15.67:0/614322974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32f31c460 con 0x55d32f428ff0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.170+0000 7feb12000640 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feb04009fd0 tx=0x7feb04012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e516f7ccfa9b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb13e00640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb13e00640 1 -- 172.21.15.67:0/3709591599 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7feb140e5d50 con 0x7feb1c113990 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb0b400640 1 -- 172.21.15.67:0/614322974 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017070 con 0x55d32f428ff0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb0b400640 1 -- 172.21.15.67:0/614322974 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb04013b50 con 0x55d32f428ff0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb1aa00640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.171+0000 7feb0b400640 1 -- 172.21.15.67:0/614322974 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0401caa0 con 0x55d32f428ff0 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 -- 172.21.15.67:0/614322974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 msgr2=0x55d32f4293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feb04009fd0 tx=0x7feb04012920 comp rx=0 tx=0).stop 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb1aa00640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 0x7feb000b7990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d32f42c170 tx=0x7feb0c032b10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 -- 172.21.15.67:0/614322974 shutdown_connections 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f4358a0 0x55d32f437c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f429910 0x55d32f435360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 --2- 172.21.15.67:0/614322974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f428ff0 0x55d32f4293d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 -- 172.21.15.67:0/614322974 >> 172.21.15.67:0/614322974 conn(0x55d32f426e20 msgr2=0x55d32f427fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 -- 172.21.15.67:0/614322974 shutdown_connections 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.172+0000 7feb241f1740 1 -- 172.21.15.67:0/614322974 wait complete. 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 -- start start 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 Processor -- start 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f428ff0 0x55d32f4327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f429910 0x55d32f432d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d32f1f0bb0 con 0x55d32f4358a0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d32f32a6c0 con 0x55d32f428ff0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d32f4385a0 con 0x55d32f429910 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.173+0000 7feb241f1740 10 client.0 _ll_register_callbacks cb 0x55d32f345490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f428ff0 0x55d32f4327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53932/0 (socket says 172.21.15.67:53932) 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 -- 172.21.15.67:0/1934094908 learned_addr learned my addr 172.21.15.67:0/1934094908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb12000640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f429910 0x55d32f432d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 -- 172.21.15.67:0/1934094908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f429910 msgr2=0x55d32f432d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f429910 0x55d32f432d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 -- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f428ff0 msgr2=0x55d32f4327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f428ff0 0x55d32f4327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 -- 172.21.15.67:0/1934094908 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32f0edb00 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feb1b400640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feb14005d00 tx=0x7feb140ab5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6ec5ee93500845 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.174+0000 7feaffe00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14020070 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7feb14091b60 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb04013b50 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb140abd90 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feb241f1740 10 client.16119 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d32f164960 con 0x55d32f4358a0 2024-09-10T23:11:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb04013b50 con 0x55d32f4358a0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7feb14057bd0 con 0x55d32f4358a0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.175+0000 7feb241f1740 10 client.16119 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb241f1740 10 client.16119 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb241f1740 20 client.16119 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4453d0 0x55d32f447830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb241f1740 4 client.16119 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d32f3a21e0 con 0x55d32f4453d0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.176+0000 7feb1aa00640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4453d0 0x55d32f447830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.177+0000 7feb1aa00640 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4453d0 0x55d32f447830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc08bc82d30054ef server_cookie=d5bb1d97883e8004 in_seq=0 out_seq=0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.177+0000 7feaffe00640 10 client.16119 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feaffe00640 1 -- 172.21.15.67:0/1934094908 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d32f3a21e0 con 0x55d32f4453d0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feaffe00640 10 client.16119 handle_command_reply: tid=0 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 client.16119 shutdown 2024-09-10T23:11:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 msgr2=0x55d32f433250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feb14005d00 tx=0x7feb140ab5c0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 shutdown_connections 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4453d0 0x55d32f447830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f4358a0 0x55d32f433250 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f429910 0x55d32f432d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 --2- 172.21.15.67:0/1934094908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f428ff0 0x55d32f4327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 >> 172.21.15.67:0/1934094908 conn(0x55d32f426e20 msgr2=0x55d32f42c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feaffe00640 0 client.16119 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 shutdown_connections 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 1 -- 172.21.15.67:0/1934094908 wait complete. 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.178+0000 7feb241f1740 20 client.16119 trim_cache size 0 max 16384 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.196+0000 7feb241f1740 1 Processor -- start 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.196+0000 7feb241f1740 1 -- start start 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f42dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f4265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b8e30 con 0x55d32f429910 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b6700 con 0x55d32f4358a0 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb241f1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b4570 con 0x55d32f428ff0 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53948/0 (socket says 172.21.15.67:53948) 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 -- 172.21.15.67:0/3651648230 learned_addr learned my addr 172.21.15.67:0/3651648230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb1aa00640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f42dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 -- 172.21.15.67:0/3651648230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 msgr2=0x55d32f42dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f42dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 -- 172.21.15.67:0/3651648230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 msgr2=0x55d32f4265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb1b400640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f4265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f4265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb12000640 1 -- 172.21.15.67:0/3651648230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32f4278a0 con 0x55d32f429910 2024-09-10T23:11:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.197+0000 7feb1b400640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f4265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.198+0000 7feb12000640 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feb04012e20 tx=0x7feb0401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f485bb091d61788e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.198+0000 7feb0aa00640 1 -- 172.21.15.67:0/3651648230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0402b070 con 0x55d32f429910 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.198+0000 7feb0aa00640 1 -- 172.21.15.67:0/3651648230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb04022040 con 0x55d32f429910 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.198+0000 7feb0aa00640 1 -- 172.21.15.67:0/3651648230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017050 con 0x55d32f429910 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- 172.21.15.67:0/3651648230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 msgr2=0x55d32f434470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feb04012e20 tx=0x7feb0401c510 comp rx=0 tx=0).stop 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- 172.21.15.67:0/3651648230 shutdown_connections 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f4265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f434470 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 --2- 172.21.15.67:0/3651648230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f42dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- 172.21.15.67:0/3651648230 >> 172.21.15.67:0/3651648230 conn(0x55d32f426e20 msgr2=0x55d32f432f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- 172.21.15.67:0/3651648230 shutdown_connections 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- 172.21.15.67:0/3651648230 wait complete. 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.199+0000 7feb241f1740 1 -- start start 2024-09-10T23:11:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 Processor -- start 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f43df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f43a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b4bc0 con 0x55d32f429910 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b6030 con 0x55d32f4358a0 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b5290 con 0x55d32f428ff0 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb241f1740 10 client.0 _ll_register_callbacks cb 0x7feb140a83e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb12000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53962/0 (socket says 172.21.15.67:53962) 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb12000640 1 -- 172.21.15.67:0/2654288976 learned_addr learned my addr 172.21.15.67:0/2654288976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:11:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.200+0000 7feb1aa00640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f43df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb1b400640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f43a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 -- 172.21.15.67:0/2654288976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 msgr2=0x55d32f43df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f43df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 -- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 msgr2=0x55d32f43a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f43a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 -- 172.21.15.67:0/2654288976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32f433670 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb12000640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb04029040 tx=0x7feb04012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5d56f322b2e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feaffe00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0402b040 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9519+0+0 (secure 0 0 0) 0x7feb04004a00 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.201+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb04022040 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb0403c050 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 10 client.16125 fetch_fsmap learned FSMap version 31 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d32f4b3090 con 0x55d32f429910 2024-09-10T23:11:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb04022040 con 0x55d32f429910 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7feb0400f040 con 0x55d32f429910 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 10 client.16125 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 10 client.16125 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 20 client.16125 populate_metadata read hostname 'smithi067' 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4b35e0 0x55d32f4b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 4 client.16125 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d30eef5", "release": true, "await": true}] 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.202+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d30eef5", "release": true, "await": true}) -- 0x55d32f39dc90 con 0x55d32f4b35e0 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.203+0000 7feb1aa00640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4b35e0 0x55d32f4b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.203+0000 7feb1aa00640 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4b35e0 0x55d32f4b5a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec63cff5686cec3 server_cookie=ba02d4bb37b03cb1 in_seq=0 out_seq=0 2024-09-10T23:11:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.203+0000 7feaffe00640 10 client.16125 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.225+0000 7feaffe00640 1 -- 172.21.15.67:0/2654288976 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d32f39dc90 con 0x55d32f4b35e0 2024-09-10T23:11:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.225+0000 7feaffe00640 10 client.16125 handle_command_reply: tid=0 2024-09-10T23:11:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.225+0000 7feb241f1740 1 client.16125 shutdown 2024-09-10T23:11:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 msgr2=0x55d32f43a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb04029040 tx=0x7feb04012f40 comp rx=0 tx=0).stop 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 shutdown_connections 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d32f4b35e0 0x55d32f4b5a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d32f4358a0 0x55d32f43a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d32f429910 0x55d32f43a420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 --2- 172.21.15.67:0/2654288976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d32f428ff0 0x55d32f43df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feaffe00640 0 client.16125 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 >> 172.21.15.67:0/2654288976 conn(0x55d32f426e20 msgr2=0x55d32f4407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 shutdown_connections 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 1 -- 172.21.15.67:0/2654288976 wait complete. 2024-09-10T23:11:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.226+0000 7feb241f1740 20 client.16125 trim_cache size 0 max 16384 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 40, 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "1d30eef5": { 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 40, 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:50.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 145.4, 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:50.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 msgr2=0x7feb000b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 0x7feb000b7990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d32f42c170 tx=0x7feb0c032b10 comp rx=0 tx=0).stop 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 msgr2=0x7feb1c1a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7feb1400d720 tx=0x7feb1400dbe0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 shutdown_connections 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb1c113990 0x7feb1c1a5090 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb1c077ff0 0x7feb1c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 --2- 172.21.15.67:0/3709591599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb1c0776d0 0x7feb1c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.227+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 >> 172.21.15.67:0/3709591599 conn(0x7feb1c100a50 msgr2=0x7feb1c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.228+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 shutdown_connections 2024-09-10T23:11:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:11:50.228+0000 7feb21600640 1 -- 172.21.15.67:0/3709591599 wait complete. 2024-09-10T23:11:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:49 smithi092 ceph-mon[34223]: pgmap v787: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T23:11:51.160 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d30eef5', total seconds elapsed: 29.5 2024-09-10T23:11:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:51 smithi067 ceph-mon[30939]: pgmap v788: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:11:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:51 smithi145 ceph-mon[34397]: pgmap v788: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:11:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:51 smithi092 ceph-mon[34223]: pgmap v788: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T23:11:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:53 smithi145 ceph-mon[34397]: pgmap v789: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:11:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:53 smithi092 ceph-mon[34223]: pgmap v789: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:11:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:53 smithi067 ceph-mon[30939]: pgmap v789: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:11:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:55 smithi145 ceph-mon[34397]: pgmap v790: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 MiB/s wr, 6 op/s 2024-09-10T23:11:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:55 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:55 smithi092 ceph-mon[34223]: pgmap v790: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 MiB/s wr, 6 op/s 2024-09-10T23:11:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:55 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:55 smithi067 ceph-mon[30939]: pgmap v790: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 MiB/s wr, 6 op/s 2024-09-10T23:11:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:55 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:57.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:11:57.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:57.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:57.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:11:57.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:57.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:57.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:11:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:57 smithi145 ceph-mon[34397]: pgmap v791: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.9 MiB/s wr, 8 op/s 2024-09-10T23:11:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:57 smithi092 ceph-mon[34223]: pgmap v791: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.9 MiB/s wr, 8 op/s 2024-09-10T23:11:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:57 smithi067 ceph-mon[30939]: pgmap v791: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.9 MiB/s wr, 8 op/s 2024-09-10T23:12:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:11:59 smithi145 ceph-mon[34397]: pgmap v792: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-10T23:12:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:11:59 smithi092 ceph-mon[34223]: pgmap v792: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-10T23:12:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:11:59 smithi067 ceph-mon[30939]: pgmap v792: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 6 op/s 2024-09-10T23:12:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:01 smithi145 ceph-mon[34397]: pgmap v793: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 10 op/s 2024-09-10T23:12:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:01 smithi092 ceph-mon[34223]: pgmap v793: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 10 op/s 2024-09-10T23:12:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:01 smithi067 ceph-mon[30939]: pgmap v793: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 10 op/s 2024-09-10T23:12:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:03 smithi145 ceph-mon[34397]: pgmap v794: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-09-10T23:12:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:03 smithi145 ceph-mon[34397]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T23:12:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:03 smithi092 ceph-mon[34223]: pgmap v794: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-09-10T23:12:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:03 smithi092 ceph-mon[34223]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T23:12:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:03 smithi067 ceph-mon[30939]: pgmap v794: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 24 op/s 2024-09-10T23:12:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:03 smithi067 ceph-mon[30939]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T23:12:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:05 smithi145 ceph-mon[34397]: pgmap v796: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:12:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:05 smithi092 ceph-mon[34223]: pgmap v796: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:12:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:05 smithi067 ceph-mon[30939]: pgmap v796: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:12:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:07 smithi145 ceph-mon[34397]: pgmap v797: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 33 op/s 2024-09-10T23:12:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:07 smithi092 ceph-mon[34223]: pgmap v797: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 33 op/s 2024-09-10T23:12:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:07 smithi067 ceph-mon[30939]: pgmap v797: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 33 op/s 2024-09-10T23:12:10.115 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:09 smithi067 ceph-mon[30939]: pgmap v798: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:12:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:09 smithi145 ceph-mon[34397]: pgmap v798: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:12:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:09 smithi092 ceph-mon[34223]: pgmap v798: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:12:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:11 smithi145 ceph-mon[34397]: pgmap v799: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:12:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:11 smithi092 ceph-mon[34223]: pgmap v799: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:12:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:11 smithi067 ceph-mon[30939]: pgmap v799: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:12:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:13 smithi145 ceph-mon[34397]: pgmap v800: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:12:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:13 smithi145 ceph-mon[34397]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T23:12:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:13 smithi092 ceph-mon[34223]: pgmap v800: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:12:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:13 smithi092 ceph-mon[34223]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T23:12:14.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:13 smithi067 ceph-mon[30939]: pgmap v800: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:12:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:13 smithi067 ceph-mon[30939]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T23:12:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:15 smithi145 ceph-mon[34397]: pgmap v802: 129 pgs: 129 active+clean; 6.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:15 smithi092 ceph-mon[34223]: pgmap v802: 129 pgs: 129 active+clean; 6.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:15 smithi067 ceph-mon[30939]: pgmap v802: 129 pgs: 129 active+clean; 6.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:17 smithi092 ceph-mon[34223]: pgmap v803: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:12:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:17 smithi067 ceph-mon[30939]: pgmap v803: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:12:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:17 smithi145 ceph-mon[34397]: pgmap v803: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:12:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:19 smithi092 ceph-mon[34223]: pgmap v804: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:20.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:19 smithi067 ceph-mon[30939]: pgmap v804: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:19 smithi145 ceph-mon[34397]: pgmap v804: 129 pgs: 129 active+clean; 6.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:21 smithi092 ceph-mon[34223]: pgmap v805: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:21 smithi067 ceph-mon[30939]: pgmap v805: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:21 smithi145 ceph-mon[34397]: pgmap v805: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:12:24.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:23 smithi092 ceph-mon[34223]: pgmap v806: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:12:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:23 smithi067 ceph-mon[30939]: pgmap v806: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:12:24.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:23 smithi145 ceph-mon[34397]: pgmap v806: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:12:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:25 smithi092 ceph-mon[34223]: pgmap v807: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:12:26.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:25 smithi067 ceph-mon[30939]: pgmap v807: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:12:26.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:25 smithi145 ceph-mon[34397]: pgmap v807: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:12:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:27 smithi092 ceph-mon[34223]: pgmap v808: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:28.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:27 smithi067 ceph-mon[30939]: pgmap v808: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:27 smithi145 ceph-mon[34397]: pgmap v808: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:29 smithi092 ceph-mon[34223]: pgmap v809: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:12:30.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:29 smithi067 ceph-mon[30939]: pgmap v809: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:12:30.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:29 smithi145 ceph-mon[34397]: pgmap v809: 129 pgs: 129 active+clean; 6.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:12:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:31 smithi092 ceph-mon[34223]: pgmap v810: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:31 smithi067 ceph-mon[30939]: pgmap v810: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:32.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:31 smithi145 ceph-mon[34397]: pgmap v810: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:33 smithi092 ceph-mon[34223]: pgmap v811: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:33 smithi067 ceph-mon[30939]: pgmap v811: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:33 smithi145 ceph-mon[34397]: pgmap v811: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:35 smithi092 ceph-mon[34223]: pgmap v812: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:36.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:35 smithi067 ceph-mon[30939]: pgmap v812: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:35 smithi145 ceph-mon[34397]: pgmap v812: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:37 smithi092 ceph-mon[34223]: pgmap v813: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:38.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:37 smithi067 ceph-mon[30939]: pgmap v813: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:38.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:37 smithi145 ceph-mon[34397]: pgmap v813: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:40.182 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:39 smithi145 ceph-mon[34397]: pgmap v814: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:12:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:39 smithi092 ceph-mon[34223]: pgmap v814: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:12:40.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:39 smithi067 ceph-mon[30939]: pgmap v814: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:12:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:41 smithi092 ceph-mon[34223]: pgmap v815: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:42.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:41 smithi067 ceph-mon[30939]: pgmap v815: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:42.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:41 smithi145 ceph-mon[34397]: pgmap v815: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:12:44.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:43 smithi092 ceph-mon[34223]: pgmap v816: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:43 smithi067 ceph-mon[30939]: pgmap v816: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:44.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:43 smithi145 ceph-mon[34397]: pgmap v816: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:12:46.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:45 smithi092 ceph-mon[34223]: pgmap v817: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:12:46.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:45 smithi067 ceph-mon[30939]: pgmap v817: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:12:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:45 smithi145 ceph-mon[34397]: pgmap v817: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:12:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:47 smithi092 ceph-mon[34223]: pgmap v818: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:47 smithi067 ceph-mon[30939]: pgmap v818: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:47 smithi145 ceph-mon[34397]: pgmap v818: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:12:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:49 smithi092 ceph-mon[34223]: pgmap v819: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:12:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:49 smithi067 ceph-mon[30939]: pgmap v819: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:12:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:49 smithi145 ceph-mon[34397]: pgmap v819: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 10 op/s 2024-09-10T23:12:52.192 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:51 smithi067 ceph-mon[30939]: pgmap v820: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:12:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:51 smithi092 ceph-mon[34223]: pgmap v820: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:12:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:51 smithi145 ceph-mon[34397]: pgmap v820: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:12:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:53 smithi092 ceph-mon[34223]: pgmap v821: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:12:54.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:53 smithi067 ceph-mon[30939]: pgmap v821: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:12:54.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:53 smithi145 ceph-mon[34397]: pgmap v821: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:12:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:55 smithi092 ceph-mon[34223]: pgmap v822: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:55 smithi067 ceph-mon[30939]: pgmap v822: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:55 smithi145 ceph-mon[34397]: pgmap v822: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:12:57.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:56 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:57.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:56 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:57.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:56 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:57 smithi092 ceph-mon[34223]: pgmap v823: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:57 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:12:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:57 smithi067 ceph-mon[30939]: pgmap v823: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:57 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:12:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:57 smithi145 ceph-mon[34397]: pgmap v823: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:12:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:57 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:13:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:12:59 smithi092 ceph-mon[34223]: pgmap v824: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:00.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:12:59 smithi067 ceph-mon[30939]: pgmap v824: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:00.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:12:59 smithi145 ceph-mon[34397]: pgmap v824: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:01 smithi092 ceph-mon[34223]: pgmap v825: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:13:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:01 smithi067 ceph-mon[30939]: pgmap v825: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:13:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:01 smithi145 ceph-mon[34397]: pgmap v825: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:13:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:03 smithi092 ceph-mon[34223]: pgmap v826: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:13:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:03 smithi092 ceph-mon[34223]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T23:13:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:03 smithi067 ceph-mon[30939]: pgmap v826: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:13:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:03 smithi067 ceph-mon[30939]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T23:13:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:03 smithi145 ceph-mon[34397]: pgmap v826: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:13:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:03 smithi145 ceph-mon[34397]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T23:13:06.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:05 smithi092 ceph-mon[34223]: pgmap v828: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:13:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:05 smithi067 ceph-mon[30939]: pgmap v828: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:13:06.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:05 smithi145 ceph-mon[34397]: pgmap v828: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 32 op/s 2024-09-10T23:13:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:07 smithi092 ceph-mon[34223]: pgmap v829: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:07 smithi067 ceph-mon[30939]: pgmap v829: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:07 smithi145 ceph-mon[34397]: pgmap v829: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:09 smithi092 ceph-mon[34223]: pgmap v830: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:13:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:09 smithi067 ceph-mon[30939]: pgmap v830: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:13:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:09 smithi145 ceph-mon[34397]: pgmap v830: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:13:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:11 smithi092 ceph-mon[34223]: pgmap v831: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:11 smithi067 ceph-mon[30939]: pgmap v831: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:12.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:11 smithi145 ceph-mon[34397]: pgmap v831: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:13:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:13 smithi092 ceph-mon[34223]: pgmap v832: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:13:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:13 smithi092 ceph-mon[34223]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T23:13:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:13 smithi067 ceph-mon[30939]: pgmap v832: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:13:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:13 smithi067 ceph-mon[30939]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T23:13:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:13 smithi145 ceph-mon[34397]: pgmap v832: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:13:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:13 smithi145 ceph-mon[34397]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T23:13:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:15 smithi092 ceph-mon[34223]: pgmap v834: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:13:16.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:15 smithi067 ceph-mon[30939]: pgmap v834: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:13:16.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:15 smithi145 ceph-mon[34397]: pgmap v834: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:13:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:17 smithi092 ceph-mon[34223]: pgmap v835: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:13:18.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:17 smithi067 ceph-mon[30939]: pgmap v835: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:13:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:17 smithi145 ceph-mon[34397]: pgmap v835: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:13:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:20 smithi092 ceph-mon[34223]: pgmap v836: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:20.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:19 smithi067 ceph-mon[30939]: pgmap v836: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:20.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:20 smithi145 ceph-mon[34397]: pgmap v836: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:13:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:22 smithi092 ceph-mon[34223]: pgmap v837: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:13:22.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:22 smithi067 ceph-mon[30939]: pgmap v837: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:13:22.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:22 smithi145 ceph-mon[34397]: pgmap v837: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:13:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:24 smithi092 ceph-mon[34223]: pgmap v838: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:13:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:24 smithi067 ceph-mon[30939]: pgmap v838: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:13:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:24 smithi145 ceph-mon[34397]: pgmap v838: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:13:25.504 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:13:25.504 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T23:13:25.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:13:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:26 smithi092 ceph-mon[34223]: pgmap v839: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:13:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:26 smithi067 ceph-mon[30939]: pgmap v839: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:13:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:26 smithi145 ceph-mon[34397]: pgmap v839: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:13:26.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 -- 172.21.15.67:0/3731296442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810a820 msgr2=0x7f8a6810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:26.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/3731296442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810a820 0x7f8a6810ac60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009900 tx=0x7f8a5402a410 comp rx=0 tx=0).stop 2024-09-10T23:13:26.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 -- 172.21.15.67:0/3731296442 shutdown_connections 2024-09-10T23:13:26.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/3731296442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810b1a0 0x7f8a681119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/3731296442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810a820 0x7f8a6810ac60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/3731296442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 0x7f8a681051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.563+0000 7f8a6ca00640 1 -- 172.21.15.67:0/3731296442 >> 172.21.15.67:0/3731296442 conn(0x7f8a68100840 msgr2=0x7f8a68102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.564+0000 7f8a6ca00640 1 -- 172.21.15.67:0/3731296442 shutdown_connections 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.564+0000 7f8a6ca00640 1 -- 172.21.15.67:0/3731296442 wait complete. 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 Processor -- start 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 -- start start 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 0x7f8a68111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810a820 0x7f8a6810e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68114ca0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68114b20 con 0x7f8a6810a820 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68114e20 con 0x7f8a68104e10 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.565+0000 7f8a67200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34618/0 (socket says 172.21.15.67:34618) 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 -- 172.21.15.67:0/4285894501 learned_addr learned my addr 172.21.15.67:0/4285894501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a65e00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810a820 0x7f8a6810e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a66800640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 0x7f8a68111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:26.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 -- 172.21.15.67:0/4285894501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 msgr2=0x7f8a68111630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 0x7f8a68111630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 -- 172.21.15.67:0/4285894501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810a820 msgr2=0x7f8a6810e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810a820 0x7f8a6810e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 -- 172.21.15.67:0/4285894501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6810f590 con 0x7f8a6810b1a0 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.566+0000 7f8a67200640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009fd0 tx=0x7f8a6000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.567+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60017070 con 0x7f8a6810b1a0 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.567+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a681ab9a0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.567+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a60003e00 con 0x7f8a6810b1a0 2024-09-10T23:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.567+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60010460 con 0x7f8a6810b1a0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.567+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681abeb0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.568+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a681051f0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.572+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8a60003970 con 0x7f8a6810b1a0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.574+0000 7f8a5f400640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 0x7f8a4c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.574+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7f8a600e5ff0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.575+0000 7f8a66800640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 0x7f8a4c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.575+0000 7f8a66800640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 0x7f8a4c0b7a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800abe0 tx=0x7f8a5800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:26.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.576+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8a600ac7f0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.855+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a6802e5c0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.856+0000 7f8a5f400640 1 -- 172.21.15.67:0/4285894501 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f8a600ac0f0 con 0x7f8a6810b1a0 2024-09-10T23:13:26.858 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:13:26.858 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:13:26.858 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:13:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.859+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 msgr2=0x7f8a4c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.859+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 0x7f8a4c0b7a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800abe0 tx=0x7f8a5800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.859+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 msgr2=0x7f8a6810edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.859+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009fd0 tx=0x7f8a6000def0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 shutdown_connections 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8a4c0b5580 0x7f8a4c0b7a40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a6810b1a0 0x7f8a6810edc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a6810a820 0x7f8a6810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 --2- 172.21.15.67:0/4285894501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a68104e10 0x7f8a68111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 >> 172.21.15.67:0/4285894501 conn(0x7f8a68100840 msgr2=0x7f8a68102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 shutdown_connections 2024-09-10T23:13:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:26.860+0000 7f8a6ca00640 1 -- 172.21.15.67:0/4285894501 wait complete. 2024-09-10T23:13:27.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:27 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4285894501' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:27.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:27 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4285894501' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:27.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:27 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4285894501' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:27.852 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T23:13:27.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T23:13:28.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:28 smithi092 ceph-mon[34223]: pgmap v840: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:13:28.283 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:28.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:28 smithi067 ceph-mon[30939]: pgmap v840: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:13:28.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:28 smithi145 ceph-mon[34397]: pgmap v840: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 -- 172.21.15.67:0/1627592416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a4104e10 msgr2=0x7f22a41051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 --2- 172.21.15.67:0/1627592416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a4104e10 0x7f22a41051f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 -- 172.21.15.67:0/1627592416 shutdown_connections 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 --2- 172.21.15.67:0/1627592416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410b1a0 0x7f22a41119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 --2- 172.21.15.67:0/1627592416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a410a820 0x7f22a410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 --2- 172.21.15.67:0/1627592416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a4104e10 0x7f22a41051f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 -- 172.21.15.67:0/1627592416 >> 172.21.15.67:0/1627592416 conn(0x7f22a4100840 msgr2=0x7f22a4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 -- 172.21.15.67:0/1627592416 shutdown_connections 2024-09-10T23:13:29.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.060+0000 7f22aa800640 1 -- 172.21.15.67:0/1627592416 wait complete. 2024-09-10T23:13:29.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.061+0000 7f22aa800640 1 Processor -- start 2024-09-10T23:13:29.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.061+0000 7f22aa800640 1 -- start start 2024-09-10T23:13:29.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410a820 0x7f22a419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 0x7f22a41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4114cd0 con 0x7f22a410b1a0 2024-09-10T23:13:29.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4114b50 con 0x7f22a410a820 2024-09-10T23:13:29.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22aa800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4114e50 con 0x7f22a4104e10 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 0x7f22a41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22a9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22a9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:55294/0 (socket says 172.21.15.67:55294) 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 0x7f22a41a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55682/0 (socket says 172.21.15.67:55682) 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.062+0000 7f22a9400640 1 -- 172.21.15.67:0/3397778723 learned_addr learned my addr 172.21.15.67:0/3397778723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a8a00640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410a820 0x7f22a419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 -- 172.21.15.67:0/3397778723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410a820 msgr2=0x7f22a419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410a820 0x7f22a419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 -- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 msgr2=0x7f22a41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 0x7f22a41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 -- 172.21.15.67:0/3397778723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41a1170 con 0x7f22a4104e10 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.063+0000 7f22a9400640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2298009230 tx=0x7f2298004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.064+0000 7f22a2000640 1 -- 172.21.15.67:0/3397778723 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229803f030 con 0x7f22a4104e10 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.064+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41a1400 con 0x7f22a4104e10 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.064+0000 7f22a2000640 1 -- 172.21.15.67:0/3397778723 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2298003a80 con 0x7f22a4104e10 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.064+0000 7f22a2000640 1 -- 172.21.15.67:0/3397778723 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298035070 con 0x7f22a4104e10 2024-09-10T23:13:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.064+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a7b20 con 0x7f22a4104e10 2024-09-10T23:13:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.067+0000 7f22aea4d740 1 Processor -- start 2024-09-10T23:13:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 -- start start 2024-09-10T23:13:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda38370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda46ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5601fd9ada90 con 0x5601fda446e0 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601fd99cec0 con 0x5601fda39e10 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22aea4d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5601fd9a0cd0 con 0x5601fda388b0 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22a2000640 1 -- 172.21.15.67:0/3397778723 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f229803bc30 con 0x7f22a4104e10 2024-09-10T23:13:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.068+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda46ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda38370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda46ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55696/0 (socket says 172.21.15.67:55696) 2024-09-10T23:13:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a8a00640 1 -- 172.21.15.67:0/155464152 learned_addr learned my addr 172.21.15.67:0/155464152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 -- 172.21.15.67:0/155464152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 msgr2=0x5601fda38cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 -- 172.21.15.67:0/155464152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 msgr2=0x5601fda46ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda46ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.069+0000 7f22a9e00640 1 -- 172.21.15.67:0/155464152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fd9429d0 con 0x5601fda39e10 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f22a9e00640 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda38370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f229c009f30 tx=0x7f229c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf78020261e27dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f2296600640 1 -- 172.21.15.67:0/155464152 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c016070 con 0x5601fda39e10 2024-09-10T23:13:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f22a2000640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 0x7f228c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f2296600640 1 -- 172.21.15.67:0/155464152 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229c004d20 con 0x5601fda39e10 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f2296600640 1 -- 172.21.15.67:0/155464152 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0126b0 con 0x5601fda39e10 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.070+0000 7f22a2000640 1 -- 172.21.15.67:0/3397778723 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7f2298106de0 con 0x7f22a4104e10 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22a9400640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 0x7f228c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 -- 172.21.15.67:0/155464152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 msgr2=0x5601fda38370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda38370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f229c009f30 tx=0x7f229c010800 comp rx=0 tx=0).stop 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 -- 172.21.15.67:0/155464152 shutdown_connections 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda46ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda38cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 --2- 172.21.15.67:0/155464152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda38370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.071+0000 7f22aea4d740 1 -- 172.21.15.67:0/155464152 >> 172.21.15.67:0/155464152 conn(0x5601fda026f0 msgr2=0x5601fda361a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.072+0000 7f22aea4d740 1 -- 172.21.15.67:0/155464152 shutdown_connections 2024-09-10T23:13:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.072+0000 7f22aea4d740 1 -- 172.21.15.67:0/155464152 wait complete. 2024-09-10T23:13:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.072+0000 7f22a9400640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 0x7f228c0b7a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5601fda3b4f0 tx=0x7f2298005b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.072+0000 7f22aea4d740 1 -- start start 2024-09-10T23:13:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 Processor -- start 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda446e0 0x5601fda42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5601fd7fe7b0 con 0x5601fda388b0 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601fd941650 con 0x5601fda39e10 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5601fda47500 con 0x5601fda446e0 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.073+0000 7f22aea4d740 10 client.0 _ll_register_callbacks cb 0x5601fd6a80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55702/0 (socket says 172.21.15.67:55702) 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a9e00640 1 -- 172.21.15.67:0/1801456243 learned_addr learned my addr 172.21.15.67:0/1801456243 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a9e00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda446e0 0x5601fda42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 -- 172.21.15.67:0/1801456243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda446e0 msgr2=0x5601fda42190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda446e0 0x5601fda42190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 -- 172.21.15.67:0/1801456243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 msgr2=0x5601fda41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a8a00640 1 -- 172.21.15.67:0/1801456243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601fd70aba0 con 0x5601fda39e10 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.074+0000 7f22a9e00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.075+0000 7f22a8a00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda41c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2290011800 tx=0x7f2290011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db442cb544993e5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.075+0000 7f228aa00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.075+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290004460 con 0x5601fda39e10 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.075+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7f229000f9a0 con 0x5601fda39e10 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.075+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c004d20 con 0x5601fda39e10 2024-09-10T23:13:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.076+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f229000fba0 con 0x5601fda39e10 2024-09-10T23:13:29.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.076+0000 7f22aea4d740 10 client.35852 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:29.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.076+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601fda5b990 con 0x5601fda39e10 2024-09-10T23:13:29.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.076+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229c004d20 con 0x5601fda39e10 2024-09-10T23:13:29.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.076+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2290018020 con 0x5601fda39e10 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 10 client.35852 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 10 client.35852 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 20 client.35852 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fda5cda0 0x5601fda5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 4 client.35852 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601fd9af9e0 con 0x5601fda5cda0 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.077+0000 7f22a9e00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fda5cda0 0x5601fda5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.078+0000 7f22a9e00640 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fda5cda0 0x5601fda5f190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb232174f275217 server_cookie=c4c88b1901295a91 in_seq=0 out_seq=0 2024-09-10T23:13:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.078+0000 7f228aa00640 10 client.35852 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f228aa00640 1 -- 172.21.15.67:0/1801456243 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601fd9af9e0 con 0x5601fda5cda0 2024-09-10T23:13:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f228aa00640 10 client.35852 handle_command_reply: tid=0 2024-09-10T23:13:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 client.35852 shutdown 2024-09-10T23:13:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 msgr2=0x5601fda41c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda41c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2290011800 tx=0x7f2290011cc0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 shutdown_connections 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fda5cda0 0x5601fda5f190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda446e0 0x5601fda42190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda41c50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 --2- 172.21.15.67:0/1801456243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda388b0 0x5601fda41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 >> 172.21.15.67:0/1801456243 conn(0x5601fda026f0 msgr2=0x5601fda3bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f228aa00640 0 client.35852 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 shutdown_connections 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.079+0000 7f22aea4d740 1 -- 172.21.15.67:0/1801456243 wait complete. 2024-09-10T23:13:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.080+0000 7f22aea4d740 20 client.35852 trim_cache size 0 max 16384 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 Processor -- start 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 -- start start 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda388b0 0x5601fda38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 0x5601fda35b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b01d0 con 0x5601fda446e0 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41adaa0 con 0x5601fda388b0 2024-09-10T23:13:29.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.099+0000 7f22aea4d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ab910 con 0x5601fda39e10 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55710/0 (socket says 172.21.15.67:55710) 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a9e00640 1 -- 172.21.15.67:0/2960600710 learned_addr learned my addr 172.21.15.67:0/2960600710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a8a00640 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 0x5601fda35b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a9e00640 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda388b0 0x5601fda38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.100+0000 7f22a8a00640 1 -- 172.21.15.67:0/2960600710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda388b0 msgr2=0x5601fda38050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f22a8a00640 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda388b0 0x5601fda38050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f22a8a00640 1 -- 172.21.15.67:0/2960600710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 msgr2=0x5601fda360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f22a8a00640 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f22a8a00640 1 -- 172.21.15.67:0/2960600710 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fda5f7f0 con 0x5601fda39e10 2024-09-10T23:13:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f22a8a00640 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 0x5601fda35b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f229001a700 tx=0x7f2290005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdbb6a703fa725d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f2295c00640 1 -- 172.21.15.67:0/2960600710 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290032070 con 0x5601fda39e10 2024-09-10T23:13:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f2295c00640 1 -- 172.21.15.67:0/2960600710 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229000f790 con 0x5601fda39e10 2024-09-10T23:13:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.101+0000 7f2295c00640 1 -- 172.21.15.67:0/2960600710 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229002a070 con 0x5601fda39e10 2024-09-10T23:13:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 -- 172.21.15.67:0/2960600710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 msgr2=0x5601fda35b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 0x5601fda35b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f229001a700 tx=0x7f2290005850 comp rx=0 tx=0).stop 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 -- 172.21.15.67:0/2960600710 shutdown_connections 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda39e10 0x5601fda35b90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 --2- 172.21.15.67:0/2960600710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda388b0 0x5601fda38050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 -- 172.21.15.67:0/2960600710 >> 172.21.15.67:0/2960600710 conn(0x5601fda026f0 msgr2=0x5601fda3ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.102+0000 7f22aea4d740 1 -- 172.21.15.67:0/2960600710 shutdown_connections 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.103+0000 7f22aea4d740 1 -- 172.21.15.67:0/2960600710 wait complete. 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.103+0000 7f22aea4d740 1 -- start start 2024-09-10T23:13:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.103+0000 7f22aea4d740 1 Processor -- start 2024-09-10T23:13:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda3d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41abf60 con 0x5601fda446e0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ad3d0 con 0x5601fda39e10 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ac630 con 0x5601fda388b0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22aea4d740 10 client.0 _ll_register_callbacks cb 0x7f2290005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda55020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:55314/0 (socket says 172.21.15.67:55314) 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a9e00640 1 -- 172.21.15.67:0/3992566148 learned_addr learned my addr 172.21.15.67:0/3992566148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59976/0 (socket says 172.21.15.67:59976) 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.104+0000 7f22a9e00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda3d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 -- 172.21.15.67:0/3992566148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 msgr2=0x5601fda55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda55020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 -- 172.21.15.67:0/3992566148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 msgr2=0x5601fda3d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda3d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 -- 172.21.15.67:0/3992566148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601fda55d60 con 0x5601fda39e10 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a9e00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda3d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f22a8a00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f229001a590 tx=0x7f229001a960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39588860e161cd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f228aa00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:13:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.105+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290032070 con 0x5601fda39e10 2024-09-10T23:13:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.106+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7f229001ac60 con 0x5601fda39e10 2024-09-10T23:13:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.106+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229000f790 con 0x5601fda39e10 2024-09-10T23:13:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.106+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f229002a050 con 0x5601fda39e10 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.106+0000 7f22aea4d740 10 client.35858 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.106+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601fda3efa0 con 0x5601fda39e10 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229000f790 con 0x5601fda39e10 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2290018070 con 0x5601fda39e10 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 10 client.35858 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 10 client.35858 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 20 client.35858 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fdacac90 0x5601fdacd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 4 client.35858 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5601fd9ab490 con 0x5601fdacac90 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.107+0000 7f22a9e00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fdacac90 0x5601fdacd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.108+0000 7f22a9e00640 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fdacac90 0x5601fdacd080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94480d763bb9eb66 server_cookie=cade100246650728 in_seq=0 out_seq=0 2024-09-10T23:13:29.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.108+0000 7f228aa00640 10 client.35858 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f228aa00640 1 -- 172.21.15.67:0/3992566148 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601fd9ab490 con 0x5601fdacac90 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f228aa00640 10 client.35858 handle_command_reply: tid=0 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 client.35858 shutdown 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 msgr2=0x5601fda55560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f229001a590 tx=0x7f229001a960 comp rx=0 tx=0).stop 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 shutdown_connections 2024-09-10T23:13:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5601fdacac90 0x5601fdacd080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5601fda446e0 0x5601fda3d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601fda39e10 0x5601fda55560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 --2- 172.21.15.67:0/3992566148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5601fda388b0 0x5601fda55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f228aa00640 0 client.35858 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 >> 172.21.15.67:0/3992566148 conn(0x5601fda026f0 msgr2=0x5601fda35b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 shutdown_connections 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.240+0000 7f22aea4d740 1 -- 172.21.15.67:0/3992566148 wait complete. 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aea4d740 20 client.35858 trim_cache size 0 max 16384 2024-09-10T23:13:29.243 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 42, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "b0572fe5": { 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 42, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:13:29.244 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.8, 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:29.245 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 msgr2=0x7f228c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 0x7f228c0b7a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5601fda3b4f0 tx=0x7f2298005b50 comp rx=0 tx=0).stop 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 msgr2=0x7f22a419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2298009230 tx=0x7f2298004270 comp rx=0 tx=0).stop 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 shutdown_connections 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f228c0b5580 0x7f228c0b7a40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a410b1a0 0x7f22a41a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.241+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a410a820 0x7f22a419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.242+0000 7f22aa800640 1 --2- 172.21.15.67:0/3397778723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22a4104e10 0x7f22a419b0c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.242+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 >> 172.21.15.67:0/3397778723 conn(0x7f22a4100840 msgr2=0x7f22a4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.242+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 shutdown_connections 2024-09-10T23:13:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:29.242+0000 7f22aa800640 1 -- 172.21.15.67:0/3397778723 wait complete. 2024-09-10T23:13:30.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b0572fe5', quiesce took 4.7 seconds. Will release after: 18.05 2024-09-10T23:13:30.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:30 smithi067 ceph-mon[30939]: pgmap v841: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:13:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:30 smithi145 ceph-mon[34397]: pgmap v841: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:13:30.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:30 smithi092 ceph-mon[34223]: pgmap v841: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:13:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:32 smithi067 ceph-mon[30939]: pgmap v842: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:13:32.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:32 smithi145 ceph-mon[34397]: pgmap v842: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:13:32.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:32 smithi092 ceph-mon[34223]: pgmap v842: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:13:34.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:34 smithi067 ceph-mon[30939]: pgmap v843: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:13:34.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:34 smithi145 ceph-mon[34397]: pgmap v843: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:13:34.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:34 smithi092 ceph-mon[34223]: pgmap v843: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:13:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:36 smithi067 ceph-mon[30939]: pgmap v844: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:13:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:36 smithi145 ceph-mon[34397]: pgmap v844: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:13:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:36 smithi092 ceph-mon[34223]: pgmap v844: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:13:38.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:38 smithi067 ceph-mon[30939]: pgmap v845: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 11 op/s 2024-09-10T23:13:38.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:38 smithi145 ceph-mon[34397]: pgmap v845: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 11 op/s 2024-09-10T23:13:38.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:38 smithi092 ceph-mon[34223]: pgmap v845: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 11 op/s 2024-09-10T23:13:40.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:40 smithi067 ceph-mon[30939]: pgmap v846: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-10T23:13:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:40 smithi145 ceph-mon[34397]: pgmap v846: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-10T23:13:40.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:40 smithi092 ceph-mon[34223]: pgmap v846: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 8 op/s 2024-09-10T23:13:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:42 smithi067 ceph-mon[30939]: pgmap v847: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 10 op/s 2024-09-10T23:13:42.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:42 smithi145 ceph-mon[34397]: pgmap v847: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 10 op/s 2024-09-10T23:13:42.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:42 smithi092 ceph-mon[34223]: pgmap v847: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 10 op/s 2024-09-10T23:13:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:44 smithi145 ceph-mon[34397]: pgmap v848: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.5 MiB/s wr, 6 op/s 2024-09-10T23:13:44.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:44 smithi092 ceph-mon[34223]: pgmap v848: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.5 MiB/s wr, 6 op/s 2024-09-10T23:13:44.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:44 smithi067 ceph-mon[30939]: pgmap v848: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.5 MiB/s wr, 6 op/s 2024-09-10T23:13:46.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:46 smithi145 ceph-mon[34397]: pgmap v849: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 849 KiB/s wr, 6 op/s 2024-09-10T23:13:46.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:46 smithi092 ceph-mon[34223]: pgmap v849: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 849 KiB/s wr, 6 op/s 2024-09-10T23:13:46.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:46 smithi067 ceph-mon[30939]: pgmap v849: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 849 KiB/s wr, 6 op/s 2024-09-10T23:13:48.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:13:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:48 smithi145 ceph-mon[34397]: pgmap v850: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 506 KiB/s wr, 7 op/s 2024-09-10T23:13:48.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:48 smithi092 ceph-mon[34223]: pgmap v850: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 506 KiB/s wr, 7 op/s 2024-09-10T23:13:48.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:48 smithi067 ceph-mon[30939]: pgmap v850: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 506 KiB/s wr, 7 op/s 2024-09-10T23:13:48.618 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 -- 172.21.15.67:0/2378086610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 msgr2=0x7f472410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 --2- 172.21.15.67:0/2378086610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f472410f950 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f471c009ca0 tx=0x7f471c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 -- 172.21.15.67:0/2378086610 shutdown_connections 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 --2- 172.21.15.67:0/2378086610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f472410f950 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 --2- 172.21.15.67:0/2378086610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 0x7f4724108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 --2- 172.21.15.67:0/2378086610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f47241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 -- 172.21.15.67:0/2378086610 >> 172.21.15.67:0/2378086610 conn(0x7f47240fe830 msgr2=0x7f4724100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 -- 172.21.15.67:0/2378086610 shutdown_connections 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.242+0000 7f4729a00640 1 -- 172.21.15.67:0/2378086610 wait complete. 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 Processor -- start 2024-09-10T23:13:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 -- start start 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f472419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 0x7f472419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4724112cd0 con 0x7f4724109150 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4724112b50 con 0x7f4724102dc0 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.243+0000 7f4729a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4724112e50 con 0x7f47241087d0 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4722a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f472419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58712/0 (socket says 172.21.15.67:58712) 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 -- 172.21.15.67:0/3389308633 learned_addr learned my addr 172.21.15.67:0/3389308633 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f471a000640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 0x7f472419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 -- 172.21.15.67:0/3389308633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 msgr2=0x7f472419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 0x7f472419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 -- 172.21.15.67:0/3389308633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 msgr2=0x7f472419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f472419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4723400640 1 -- 172.21.15.67:0/3389308633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47241a1430 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.244+0000 7f4722a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f472419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f4723400640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f471c005cc0 tx=0x7f471c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c032e00 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f471c0044d0 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c030b70 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241a16c0 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.245+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47241a7de0 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.246+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47241031a0 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.249+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f471c004680 con 0x7f4724109150 2024-09-10T23:13:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.250+0000 7f471be00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 0x7f47080b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.250+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7f471c1076f0 con 0x7f4724109150 2024-09-10T23:13:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.250+0000 7f4722a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 0x7f47080b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.251+0000 7f4722a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 0x7f47080b7990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4714005d00 tx=0x7f471400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.252+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f471c0cde30 con 0x7f4724109150 2024-09-10T23:13:49.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.493+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f472402e5c0 con 0x7f4724109150 2024-09-10T23:13:49.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.494+0000 7f471be00640 1 -- 172.21.15.67:0/3389308633 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f471c0cd730 con 0x7f4724109150 2024-09-10T23:13:49.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:13:49.496 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:13:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 msgr2=0x7f47080b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 0x7f47080b7990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4714005d00 tx=0x7f471400b410 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 msgr2=0x7f47241a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f471c005cc0 tx=0x7f471c032a10 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 shutdown_connections 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f47080b54d0 0x7f47080b7990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4724109150 0x7f47241a0c60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47241087d0 0x7f472419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 --2- 172.21.15.67:0/3389308633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4724102dc0 0x7f472419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 >> 172.21.15.67:0/3389308633 conn(0x7f47240fe830 msgr2=0x7f4724106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 shutdown_connections 2024-09-10T23:13:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:49.497+0000 7f4729a00640 1 -- 172.21.15.67:0/3389308633 wait complete. 2024-09-10T23:13:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:50 smithi067 ceph-mon[30939]: pgmap v851: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:13:50.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:50 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3389308633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:50.323 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:50 smithi092 ceph-mon[34223]: pgmap v851: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:13:50.323 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:50 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3389308633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:50 smithi145 ceph-mon[34397]: pgmap v851: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:13:50.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:50 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3389308633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:50.425 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id b0572fe5 --release --await' 2024-09-10T23:13:50.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id b0572fe5 --release --await 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:51.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 -- 172.21.15.67:0/104130875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075540 msgr2=0x7fe720075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 --2- 172.21.15.67:0/104130875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075540 0x7fe720075920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe710009900 tx=0x7fe71002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 -- 172.21.15.67:0/104130875 shutdown_connections 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 --2- 172.21.15.67:0/104130875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe72010f3e0 0x7fe7201117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 --2- 172.21.15.67:0/104130875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe720075f40 0x7fe72010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 --2- 172.21.15.67:0/104130875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075540 0x7fe720075920 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 -- 172.21.15.67:0/104130875 >> 172.21.15.67:0/104130875 conn(0x7fe7200fe830 msgr2=0x7fe720100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 -- 172.21.15.67:0/104130875 shutdown_connections 2024-09-10T23:13:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.392+0000 7fe725a00640 1 -- 172.21.15.67:0/104130875 wait complete. 2024-09-10T23:13:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.393+0000 7fe725a00640 1 Processor -- start 2024-09-10T23:13:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.393+0000 7fe725a00640 1 -- start start 2024-09-10T23:13:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe720075540 0x7fe72019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe72010f3e0 0x7fe7201a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe720114ea0 con 0x7fe720075f40 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe720114d20 con 0x7fe72010f3e0 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe725a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe720115020 con 0x7fe720075540 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe71e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe71e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58720/0 (socket says 172.21.15.67:58720) 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe71e000640 1 -- 172.21.15.67:0/3910983914 learned_addr learned my addr 172.21.15.67:0/3910983914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe71ea00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe720075540 0x7fe72019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.394+0000 7fe71f400640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe72010f3e0 0x7fe7201a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 -- 172.21.15.67:0/3910983914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe720075540 msgr2=0x7fe72019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe720075540 0x7fe72019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 -- 172.21.15.67:0/3910983914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe72010f3e0 msgr2=0x7fe7201a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe72010f3e0 0x7fe7201a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 -- 172.21.15.67:0/3910983914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7201a1460 con 0x7fe720075f40 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe71e000640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe717400640 1 -- 172.21.15.67:0/3910983914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c017070 con 0x7fe720075f40 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe717400640 1 -- 172.21.15.67:0/3910983914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe70c009c30 con 0x7fe720075f40 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7201a1750 con 0x7fe720075f40 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe717400640 1 -- 172.21.15.67:0/3910983914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00f620 con 0x7fe720075f40 2024-09-10T23:13:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.395+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7201a7e10 con 0x7fe720075f40 2024-09-10T23:13:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.398+0000 7fe728426740 1 Processor -- start 2024-09-10T23:13:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 -- start start 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd42910 0x557eabd4e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd4e8a0 0x557eabd50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557eabcb9290 con 0x557eabd41ff0 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557eabca86c0 con 0x557eabd42910 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe728426740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557eabcac4d0 con 0x557eabd4e8a0 2024-09-10T23:13:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe71ea00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd42910 0x557eabd4e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58722/0 (socket says 172.21.15.67:58722) 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.399+0000 7fe71f400640 1 -- 172.21.15.67:0/795379335 learned_addr learned my addr 172.21.15.67:0/795379335 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 -- 172.21.15.67:0/795379335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd4e8a0 msgr2=0x557eabd50c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd4e8a0 0x557eabd50c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 -- 172.21.15.67:0/795379335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd42910 msgr2=0x557eabd4e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd42910 0x557eabd4e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 -- 172.21.15.67:0/795379335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eabc35460 con 0x557eabd41ff0 2024-09-10T23:13:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe717400640 1 -- 172.21.15.67:0/3910983914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe70c010460 con 0x7fe720075f40 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe71f400640 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe718009fc0 tx=0x7fe7180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1767f766f7d3efa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe70be00640 1 -- 172.21.15.67:0/795379335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718004b20 con 0x557eabd41ff0 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe70be00640 1 -- 172.21.15.67:0/795379335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe718004cc0 con 0x557eabd41ff0 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.400+0000 7fe70be00640 1 -- 172.21.15.67:0/795379335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718004840 con 0x557eabd41ff0 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 -- 172.21.15.67:0/795379335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 msgr2=0x557eabd423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe718009fc0 tx=0x7fe7180118d0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 -- 172.21.15.67:0/795379335 shutdown_connections 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd4e8a0 0x557eabd50c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd42910 0x557eabd4e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 --2- 172.21.15.67:0/795379335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd41ff0 0x557eabd423d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 -- 172.21.15.67:0/795379335 >> 172.21.15.67:0/795379335 conn(0x557eabd3fe20 msgr2=0x557eabd40fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 -- 172.21.15.67:0/795379335 shutdown_connections 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe717400640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 0x7fe7040b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.401+0000 7fe728426740 1 -- 172.21.15.67:0/795379335 wait complete. 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.402+0000 7fe717400640 1 -- 172.21.15.67:0/3910983914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fe70c0e5f50 con 0x7fe720075f40 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.402+0000 7fe71ea00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 0x7fe7040b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.402+0000 7fe728426740 1 -- start start 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.402+0000 7fe728426740 1 Processor -- start 2024-09-10T23:13:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.402+0000 7fe71ea00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 0x7fe7040b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd4b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557eabb09bb0 con 0x557eabd4e8a0 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557eabc436c0 con 0x557eabd41ff0 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557eabd51550 con 0x557eabd42910 2024-09-10T23:13:51.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe728426740 10 client.0 _ll_register_callbacks cb 0x557eabc5e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58730/0 (socket says 172.21.15.67:58730) 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe71e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe71f400640 1 -- 172.21.15.67:0/764217356 learned_addr learned my addr 172.21.15.67:0/764217356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.403+0000 7fe71ea00640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd4b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 -- 172.21.15.67:0/764217356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 msgr2=0x557eabd4be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd4be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 -- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 msgr2=0x557eabd4b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd4b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 -- 172.21.15.67:0/764217356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eaba06b00 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe71f400640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe718004440 tx=0x7fe718009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61b73879010733e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe708c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718026020 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe718004cc0 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.404+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fe718003dc0 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe718031050 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 10 client.16167 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557eaba7d960 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe718004cc0 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe718005000 con 0x557eabd4e8a0 2024-09-10T23:13:51.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 10 client.16167 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 10 client.16167 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 20 client.16167 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabd66bb0 0x557eabd68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 4 client.16167 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.405+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eabcbb1e0 con 0x557eabd66bb0 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.406+0000 7fe71ea00640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabd66bb0 0x557eabd68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.407+0000 7fe71ea00640 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabd66bb0 0x557eabd68fa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ec4178d3cc8e7c server_cookie=dbef9fb2b20034d5 in_seq=0 out_seq=0 2024-09-10T23:13:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.407+0000 7fe708c00640 10 client.16167 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.407+0000 7fe708c00640 1 -- 172.21.15.67:0/764217356 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557eabcbb1e0 con 0x557eabd66bb0 2024-09-10T23:13:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.407+0000 7fe708c00640 10 client.16167 handle_command_reply: tid=0 2024-09-10T23:13:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 client.16167 shutdown 2024-09-10T23:13:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 msgr2=0x557eabd4c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe718004440 tx=0x7fe718009d20 comp rx=0 tx=0).stop 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 shutdown_connections 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabd66bb0 0x557eabd68fa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd4c350 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd4be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 --2- 172.21.15.67:0/764217356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd4b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 >> 172.21.15.67:0/764217356 conn(0x557eabd3fe20 msgr2=0x557eabd45990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe708c00640 0 client.16167 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 shutdown_connections 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 1 -- 172.21.15.67:0/764217356 wait complete. 2024-09-10T23:13:51.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.408+0000 7fe728426740 20 client.16167 trim_cache size 0 max 16384 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.426+0000 7fe728426740 1 Processor -- start 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.426+0000 7fe728426740 1 -- start start 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd48ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201b0580 con 0x557eabd4e8a0 2024-09-10T23:13:51.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201ade50 con 0x557eabd41ff0 2024-09-10T23:13:51.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe728426740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201abd00 con 0x557eabd42910 2024-09-10T23:13:51.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58738/0 (socket says 172.21.15.67:58738) 2024-09-10T23:13:51.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 -- 172.21.15.67:0/1566352675 learned_addr learned my addr 172.21.15.67:0/1566352675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:51.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71ea00640 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71e000640 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd48ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 -- 172.21.15.67:0/1566352675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 msgr2=0x557eabd48ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd48ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 -- 172.21.15.67:0/1566352675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 msgr2=0x557eabd460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.427+0000 7fe71f400640 1 -- 172.21.15.67:0/1566352675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eabd408a0 con 0x557eabd4e8a0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.428+0000 7fe71f400640 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe7180044f0 tx=0x7fe718005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd9b798bde5e3be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.428+0000 7fe70b400640 1 -- 172.21.15.67:0/1566352675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718026070 con 0x557eabd4e8a0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.428+0000 7fe70b400640 1 -- 172.21.15.67:0/1566352675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe718015070 con 0x557eabd4e8a0 2024-09-10T23:13:51.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.428+0000 7fe70b400640 1 -- 172.21.15.67:0/1566352675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180041e0 con 0x557eabd4e8a0 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- 172.21.15.67:0/1566352675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 msgr2=0x557eabd49020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe7180044f0 tx=0x7fe718005fd0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- 172.21.15.67:0/1566352675 shutdown_connections 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd49020 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd48ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 --2- 172.21.15.67:0/1566352675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- 172.21.15.67:0/1566352675 >> 172.21.15.67:0/1566352675 conn(0x557eabd3fe20 msgr2=0x557eabd4cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- 172.21.15.67:0/1566352675 shutdown_connections 2024-09-10T23:13:51.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- 172.21.15.67:0/1566352675 wait complete. 2024-09-10T23:13:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.429+0000 7fe728426740 1 -- start start 2024-09-10T23:13:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 Processor -- start 2024-09-10T23:13:51.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201ac350 con 0x557eabd4e8a0 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201ad780 con 0x557eabd41ff0 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201ac9e0 con 0x557eabd42910 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe728426740 10 client.0 _ll_register_callbacks cb 0x7fe710009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe71f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58740/0 (socket says 172.21.15.67:58740) 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe71f400640 1 -- 172.21.15.67:0/757564850 learned_addr learned my addr 172.21.15.67:0/757564850 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:51.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.430+0000 7fe71e000640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71ea00640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 -- 172.21.15.67:0/757564850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 msgr2=0x557eabd5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd5fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 -- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 msgr2=0x557eabd5f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd5f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 -- 172.21.15.67:0/757564850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eabd5c4e0 con 0x557eabd4e8a0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe71f400640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe7180049e0 tx=0x7fe718011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ac77c4a724811b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe708c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.431+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718042060 con 0x557eabd4e8a0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fe718005610 con 0x557eabd4e8a0 2024-09-10T23:13:51.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe718015070 con 0x557eabd4e8a0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe718049050 con 0x557eabd4e8a0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 10 client.16173 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557eabdd4700 con 0x557eabd4e8a0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe718015070 con 0x557eabd4e8a0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe71800f750 con 0x557eabd4e8a0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 10 client.16173 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 10 client.16173 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 20 client.16173 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabdd4dd0 0x557eabdd71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 4 client.16173 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b0572fe5", "release": true, "await": true}] 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.432+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b0572fe5", "release": true, "await": true}) -- 0x557eabcb6c90 con 0x557eabdd4dd0 2024-09-10T23:13:51.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.433+0000 7fe71ea00640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabdd4dd0 0x557eabdd71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:51.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.433+0000 7fe71ea00640 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabdd4dd0 0x557eabdd71f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35921dddc185f66c server_cookie=62405d52d65dde5f in_seq=0 out_seq=0 2024-09-10T23:13:51.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.433+0000 7fe708c00640 10 client.16173 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:51.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.461+0000 7fe708c00640 1 -- 172.21.15.67:0/757564850 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557eabcb6c90 con 0x557eabdd4dd0 2024-09-10T23:13:51.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.461+0000 7fe708c00640 10 client.16173 handle_command_reply: tid=0 2024-09-10T23:13:51.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.461+0000 7fe728426740 1 client.16173 shutdown 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.461+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 msgr2=0x557eabd5bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.461+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe7180049e0 tx=0x7fe718011e50 comp rx=0 tx=0).stop 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 shutdown_connections 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557eabdd4dd0 0x557eabdd71f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557eabd4e8a0 0x557eabd5bcb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe708c00640 0 client.16173 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557eabd42910 0x557eabd5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 --2- 172.21.15.67:0/757564850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557eabd41ff0 0x557eabd5f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 >> 172.21.15.67:0/757564850 conn(0x557eabd3fe20 msgr2=0x557eabd4b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 shutdown_connections 2024-09-10T23:13:51.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 1 -- 172.21.15.67:0/757564850 wait complete. 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.462+0000 7fe728426740 20 client.16173 trim_cache size 0 max 16384 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 44, 2024-09-10T23:13:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "b0572fe5": { 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 44, 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.8, 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:13:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:51.467 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:13:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 msgr2=0x7fe7040b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 0x7fe7040b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710005e80 comp rx=0 tx=0).stop 2024-09-10T23:13:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 msgr2=0x7fe72019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00de10 comp rx=0 tx=0).stop 2024-09-10T23:13:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 shutdown_connections 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe7040b5360 0x7fe7040b7820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe72010f3e0 0x7fe7201a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720075f40 0x7fe72019b8f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 --2- 172.21.15.67:0/3910983914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe720075540 0x7fe72019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 >> 172.21.15.67:0/3910983914 conn(0x7fe7200fe830 msgr2=0x7fe7200ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 shutdown_connections 2024-09-10T23:13:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:51.463+0000 7fe725a00640 1 -- 172.21.15.67:0/3910983914 wait complete. 2024-09-10T23:13:52.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:52 smithi145 ceph-mon[34397]: pgmap v852: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:13:52.437 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:52 smithi067 ceph-mon[30939]: pgmap v852: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:13:52.478 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b0572fe5', total seconds elapsed: 27.0 2024-09-10T23:13:52.478 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T23:13:52.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:13:52.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:52 smithi092 ceph-mon[34223]: pgmap v852: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T23:13:52.897 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 -- 172.21.15.67:0/2017966901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 msgr2=0x7fc874105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2017966901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc874105380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc868009900 tx=0x7fc86802a410 comp rx=0 tx=0).stop 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 -- 172.21.15.67:0/2017966901 shutdown_connections 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2017966901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410b330 0x7fc874111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2017966901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410a9b0 0x7fc87410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2017966901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc874105380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 -- 172.21.15.67:0/2017966901 >> 172.21.15.67:0/2017966901 conn(0x7fc874100a50 msgr2=0x7fc874102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 -- 172.21.15.67:0/2017966901 shutdown_connections 2024-09-10T23:13:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.586+0000 7fc87ba00640 1 -- 172.21.15.67:0/2017966901 wait complete. 2024-09-10T23:13:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.587+0000 7fc87ba00640 1 Processor -- start 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 -- start start 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410a9b0 0x7fc87419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410b330 0x7fc8741a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc874114e60 con 0x7fc874104fa0 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc874114ce0 con 0x7fc87410b330 2024-09-10T23:13:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.588+0000 7fc87ba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc874114fe0 con 0x7fc87410a9b0 2024-09-10T23:13:53.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:53.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58760/0 (socket says 172.21.15.67:58760) 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 -- 172.21.15.67:0/2308411543 learned_addr learned my addr 172.21.15.67:0/2308411543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc873e00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410a9b0 0x7fc87419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879a00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410b330 0x7fc8741a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 -- 172.21.15.67:0/2308411543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410a9b0 msgr2=0x7fc87419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410a9b0 0x7fc87419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 -- 172.21.15.67:0/2308411543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410b330 msgr2=0x7fc8741a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410b330 0x7fc8741a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.589+0000 7fc879000640 1 -- 172.21.15.67:0/2308411543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8741a1520 con 0x7fc874104fa0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc879000640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc86802a930 tx=0x7fc868003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86803f070 con 0x7fc874104fa0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc868003e40 con 0x7fc874104fa0 2024-09-10T23:13:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8741a17b0 con 0x7fc874104fa0 2024-09-10T23:13:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868035030 con 0x7fc874104fa0 2024-09-10T23:13:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.590+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8741a7ed0 con 0x7fc874104fa0 2024-09-10T23:13:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.591+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc874105380 con 0x7fc874104fa0 2024-09-10T23:13:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.594+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc868031400 con 0x7fc874104fa0 2024-09-10T23:13:53.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.596+0000 7fc871600640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 0x7fc85c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:53.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.596+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fc868106e90 con 0x7fc874104fa0 2024-09-10T23:13:53.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.597+0000 7fc873e00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 0x7fc85c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:53.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.597+0000 7fc873e00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 0x7fc85c0b7990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc87410a150 tx=0x7fc86000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:53.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.598+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc8680cd5d0 con 0x7fc874104fa0 2024-09-10T23:13:53.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.889+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc87402e5c0 con 0x7fc874104fa0 2024-09-10T23:13:53.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.890+0000 7fc871600640 1 -- 172.21.15.67:0/2308411543 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fc8680cced0 con 0x7fc874104fa0 2024-09-10T23:13:53.892 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:13:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:13:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.892+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 msgr2=0x7fc85c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.892+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 0x7fc85c0b7990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc87410a150 tx=0x7fc86000b390 comp rx=0 tx=0).stop 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.892+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 msgr2=0x7fc87419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.892+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc86802a930 tx=0x7fc868003c20 comp rx=0 tx=0).stop 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 shutdown_connections 2024-09-10T23:13:53.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc85c0b54d0 0x7fc85c0b7990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87410b330 0x7fc8741a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc87410a9b0 0x7fc87419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 --2- 172.21.15.67:0/2308411543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc874104fa0 0x7fc87419b470 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 >> 172.21.15.67:0/2308411543 conn(0x7fc874100a50 msgr2=0x7fc8741014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 shutdown_connections 2024-09-10T23:13:53.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:53.893+0000 7fc87ba00640 1 -- 172.21.15.67:0/2308411543 wait complete. 2024-09-10T23:13:54.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:54 smithi067 ceph-mon[30939]: pgmap v853: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:13:54.265 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:54 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2308411543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:54.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:54 smithi145 ceph-mon[34397]: pgmap v853: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:13:54.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:54 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2308411543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:54.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:54 smithi092 ceph-mon[34223]: pgmap v853: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T23:13:54.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:54 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2308411543' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:54.679 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T23:13:54.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T23:13:55.134 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 -- 172.21.15.67:0/623211759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c0754e0 msgr2=0x7fc09c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 --2- 172.21.15.67:0/623211759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c0754e0 0x7fc09c0758c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009900 tx=0x7fc08c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 -- 172.21.15.67:0/623211759 shutdown_connections 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 --2- 172.21.15.67:0/623211759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c10f200 0x7fc09c1115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 --2- 172.21.15.67:0/623211759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c075e00 0x7fc09c10ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 --2- 172.21.15.67:0/623211759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c0754e0 0x7fc09c0758c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.783+0000 7fc09b200640 1 -- 172.21.15.67:0/623211759 >> 172.21.15.67:0/623211759 conn(0x7fc09c0fe660 msgr2=0x7fc09c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.784+0000 7fc09b200640 1 -- 172.21.15.67:0/623211759 shutdown_connections 2024-09-10T23:13:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.784+0000 7fc09b200640 1 -- 172.21.15.67:0/623211759 wait complete. 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.784+0000 7fc09b200640 1 Processor -- start 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 -- start start 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 0x7fc09c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 0x7fc09c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c10f200 0x7fc09c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c114ce0 con 0x7fc09c075e00 2024-09-10T23:13:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c114b60 con 0x7fc09c10f200 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc09b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c114e60 con 0x7fc09c0754e0 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 0x7fc09c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc099e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 0x7fc09c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.785+0000 7fc099e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 0x7fc09c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:34174/0 (socket says 172.21.15.67:34174) 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099e00640 1 -- 172.21.15.67:0/2431078656 learned_addr learned my addr 172.21.15.67:0/2431078656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc09a800640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c10f200 0x7fc09c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 -- 172.21.15.67:0/2431078656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 msgr2=0x7fc09c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 0x7fc09c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 -- 172.21.15.67:0/2431078656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c10f200 msgr2=0x7fc09c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c10f200 0x7fc09c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 -- 172.21.15.67:0/2431078656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c1a1260 con 0x7fc09c075e00 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.786+0000 7fc099400640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 0x7fc09c19b6f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc08800e9f0 tx=0x7fc08800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.787+0000 7fc092a00640 1 -- 172.21.15.67:0/2431078656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08800ccc0 con 0x7fc09c075e00 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.787+0000 7fc092a00640 1 -- 172.21.15.67:0/2431078656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0880108c0 con 0x7fc09c075e00 2024-09-10T23:13:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.787+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c1a1550 con 0x7fc09c075e00 2024-09-10T23:13:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.787+0000 7fc092a00640 1 -- 172.21.15.67:0/2431078656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880187e0 con 0x7fc09c075e00 2024-09-10T23:13:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.787+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1a7c10 con 0x7fc09c075e00 2024-09-10T23:13:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.790+0000 7fc0a3a79740 1 Processor -- start 2024-09-10T23:13:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.790+0000 7fc0a3a79740 1 -- start start 2024-09-10T23:13:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ebbe10 0x559141eba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141ec8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559141e2fa90 con 0x559141ec66e0 2024-09-10T23:13:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559141e1eec0 con 0x559141ebbe10 2024-09-10T23:13:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559141e22cd0 con 0x559141eba8b0 2024-09-10T23:13:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc099e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc099e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:34178/0 (socket says 172.21.15.67:34178) 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc099e00640 1 -- 172.21.15.67:0/3406632046 learned_addr learned my addr 172.21.15.67:0/3406632046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.791+0000 7fc09a800640 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ebbe10 0x559141eba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 -- 172.21.15.67:0/3406632046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ebbe10 msgr2=0x559141eba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ebbe10 0x559141eba370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 -- 172.21.15.67:0/3406632046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 msgr2=0x559141ec8ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141ec8ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 -- 172.21.15.67:0/3406632046 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559141dc49d0 con 0x559141eba8b0 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc092a00640 1 -- 172.21.15.67:0/2431078656 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc088010430 con 0x7fc09c075e00 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc099e00640 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc08c004070 tx=0x7fc08c0040a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a309e723325b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc087400640 1 -- 172.21.15.67:0/3406632046 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c040070 con 0x559141eba8b0 2024-09-10T23:13:55.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc087400640 1 -- 172.21.15.67:0/3406632046 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc08c0054b0 con 0x559141eba8b0 2024-09-10T23:13:55.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.792+0000 7fc087400640 1 -- 172.21.15.67:0/3406632046 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c035040 con 0x559141eba8b0 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3406632046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 msgr2=0x559141ebacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc08c004070 tx=0x7fc08c0040a0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3406632046 shutdown_connections 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141ec8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ebacd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3406632046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ebbe10 0x559141eba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.793+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3406632046 >> 172.21.15.67:0/3406632046 conn(0x559141e846f0 msgr2=0x559141eb81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3406632046 shutdown_connections 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3406632046 wait complete. 2024-09-10T23:13:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc092a00640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 0x7fc0800b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc092a00640 1 -- 172.21.15.67:0/2431078656 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fc0880e6860 con 0x7fc09c075e00 2024-09-10T23:13:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc0a3a79740 1 -- start start 2024-09-10T23:13:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc099e00640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 0x7fc0800b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.794+0000 7fc0a3a79740 1 Processor -- start 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141ec3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc099e00640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 0x7fc0800b78d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc08c02a910 tx=0x7fc08c000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ec4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559141c807b0 con 0x559141eba8b0 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559141dc3650 con 0x559141ec66e0 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559141ec94a0 con 0x559141ebbe10 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.795+0000 7fc0a3a79740 10 client.0 _ll_register_callbacks cb 0x559141b2a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58794/0 (socket says 172.21.15.67:58794) 2024-09-10T23:13:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 -- 172.21.15.67:0/3640701981 learned_addr learned my addr 172.21.15.67:0/3640701981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc09a800640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ec4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099400640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141ec3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 -- 172.21.15.67:0/3640701981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 msgr2=0x559141ec3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141ec3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 -- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 msgr2=0x559141ec4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ec4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.796+0000 7fc099e00640 1 -- 172.21.15.67:0/3640701981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559141b8cba0 con 0x559141eba8b0 2024-09-10T23:13:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc099e00640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc08c043000 tx=0x7fc08c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce7f2b60c40b081 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc07be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:13:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c057070 con 0x559141eba8b0 2024-09-10T23:13:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08c0054b0 con 0x559141eba8b0 2024-09-10T23:13:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fc08c003a00 con 0x559141eba8b0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc08c040070 con 0x559141eba8b0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc0a3a79740 10 client.16188 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.797+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559141ed5040 con 0x559141eba8b0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc08c0054b0 con 0x559141eba8b0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc08c04c040 con 0x559141eba8b0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 10 client.16188 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 10 client.16188 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 20 client.16188 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141ed6390 0x559141ed8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 4 client.16188 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559141e319e0 con 0x559141ed6390 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.798+0000 7fc099400640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141ed6390 0x559141ed8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.799+0000 7fc07be00640 10 client.16188 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.799+0000 7fc099400640 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141ed6390 0x559141ed8780 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1e99dfb39897df server_cookie=4d04833c92773a96 in_seq=0 out_seq=0 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc07be00640 1 -- 172.21.15.67:0/3640701981 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559141e319e0 con 0x559141ed6390 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc07be00640 10 client.16188 handle_command_reply: tid=0 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 client.16188 shutdown 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 msgr2=0x559141ec3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc08c043000 tx=0x7fc08c009260 comp rx=0 tx=0).stop 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 shutdown_connections 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141ed6390 0x559141ed8780 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ec4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141ec3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/3640701981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141eba8b0 0x559141ec3680 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 >> 172.21.15.67:0/3640701981 conn(0x559141e846f0 msgr2=0x559141ebdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.800+0000 7fc07be00640 0 client.16188 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:55.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.801+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 shutdown_connections 2024-09-10T23:13:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.801+0000 7fc0a3a79740 1 -- 172.21.15.67:0/3640701981 wait complete. 2024-09-10T23:13:55.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.801+0000 7fc0a3a79740 20 client.16188 trim_cache size 0 max 16384 2024-09-10T23:13:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 Processor -- start 2024-09-10T23:13:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 -- start start 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 0x559141ec5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141eb7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b47e0 con 0x559141ec66e0 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b20b0 con 0x559141eba8b0 2024-09-10T23:13:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.820+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1aff20 con 0x559141ebbe10 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc099e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 0x559141ec5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc099400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141eb7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58810/0 (socket says 172.21.15.67:58810) 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 -- 172.21.15.67:0/2223764213 learned_addr learned my addr 172.21.15.67:0/2223764213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 -- 172.21.15.67:0/2223764213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 msgr2=0x559141eb7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141eb7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 -- 172.21.15.67:0/2223764213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 msgr2=0x559141ec5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 0x559141ec5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc09a800640 1 -- 172.21.15.67:0/2223764213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559141ed2620 con 0x559141ec66e0 2024-09-10T23:13:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc099400640 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141eb7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.821+0000 7fc099e00640 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 0x559141ec5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.822+0000 7fc09a800640 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc094003a70 tx=0x7fc094003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7607f29e311e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.822+0000 7fc086a00640 1 -- 172.21.15.67:0/2223764213 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094004660 con 0x559141ec66e0 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.822+0000 7fc086a00640 1 -- 172.21.15.67:0/2223764213 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc094004800 con 0x559141ec66e0 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.822+0000 7fc086a00640 1 -- 172.21.15.67:0/2223764213 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09401b8a0 con 0x559141ec66e0 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- 172.21.15.67:0/2223764213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 msgr2=0x559141eb7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc094003a70 tx=0x7fc094003e70 comp rx=0 tx=0).stop 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- 172.21.15.67:0/2223764213 shutdown_connections 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ec66e0 0x559141eb7ff0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141ebbe10 0x559141eb7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/2223764213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141eba8b0 0x559141ec5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- 172.21.15.67:0/2223764213 >> 172.21.15.67:0/2223764213 conn(0x559141e846f0 msgr2=0x559141ed1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- 172.21.15.67:0/2223764213 shutdown_connections 2024-09-10T23:13:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- 172.21.15.67:0/2223764213 wait complete. 2024-09-10T23:13:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.823+0000 7fc0a3a79740 1 -- start start 2024-09-10T23:13:55.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 Processor -- start 2024-09-10T23:13:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ec0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ececa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b0570 con 0x559141ebbe10 2024-09-10T23:13:55.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b19e0 con 0x559141ec66e0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b0c40 con 0x559141eba8b0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc0a3a79740 10 client.0 _ll_register_callbacks cb 0x7fc094009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58814/0 (socket says 172.21.15.67:58814) 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.824+0000 7fc099400640 1 -- 172.21.15.67:0/752740626 learned_addr learned my addr 172.21.15.67:0/752740626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc09a800640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ececa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099e00640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ec0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 -- 172.21.15.67:0/752740626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 msgr2=0x559141ec0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ec0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 -- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 msgr2=0x559141ececa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ececa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 -- 172.21.15.67:0/752740626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559141ecf3e0 con 0x559141ebbe10 2024-09-10T23:13:55.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc09a800640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ececa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc099400640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ac650 tx=0x7fc0880a8920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cf026efe183547 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc07be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880d3020 con 0x559141ebbe10 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.825+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9519+0+0 (secure 0 0 0) 0x7fc0880b59a0 con 0x559141ebbe10 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc094004800 con 0x559141ebbe10 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc0880b5e30 con 0x559141ebbe10 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc0a3a79740 10 client.16194 fetch_fsmap learned FSMap version 31 2024-09-10T23:13:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559141f43fc0 con 0x559141ebbe10 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc094004800 con 0x559141ebbe10 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.826+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc0880e64c0 con 0x559141ebbe10 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 10 client.16194 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 10 client.16194 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 20 client.16194 populate_metadata read hostname 'smithi067' 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141f44780 0x559141f46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 4 client.16194 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x559141e2d490 con 0x559141f44780 2024-09-10T23:13:55.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.827+0000 7fc099e00640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141f44780 0x559141f46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:55.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.828+0000 7fc099e00640 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141f44780 0x559141f46ba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83555fd8a6bbd057 server_cookie=cce0a4d7ad4d4f6f in_seq=0 out_seq=0 2024-09-10T23:13:55.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.828+0000 7fc07be00640 10 client.16194 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc07be00640 1 -- 172.21.15.67:0/752740626 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559141e2d490 con 0x559141f44780 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc07be00640 10 client.16194 handle_command_reply: tid=0 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 client.16194 shutdown 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 msgr2=0x559141ece760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ac650 tx=0x7fc0880a8920 comp rx=0 tx=0).stop 2024-09-10T23:13:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 shutdown_connections 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559141f44780 0x559141f46ba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559141ec66e0 0x559141ececa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559141ebbe10 0x559141ece760 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 --2- 172.21.15.67:0/752740626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559141eba8b0 0x559141ec0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 >> 172.21.15.67:0/752740626 conn(0x559141e846f0 msgr2=0x559141ec1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc07be00640 0 client.16194 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 shutdown_connections 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 1 -- 172.21.15.67:0/752740626 wait complete. 2024-09-10T23:13:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.958+0000 7fc0a3a79740 20 client.16194 trim_cache size 0 max 16384 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 46, 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "b0b26892": { 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 46, 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:55.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.8, 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:13:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 msgr2=0x7fc0800b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 0x7fc0800b78d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc08c02a910 tx=0x7fc08c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 msgr2=0x7fc09c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 0x7fc09c19b6f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc08800e9f0 tx=0x7fc08800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 shutdown_connections 2024-09-10T23:13:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc0800b5410 0x7fc0800b78d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc09c10f200 0x7fc09c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc09c075e00 0x7fc09c19b6f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 --2- 172.21.15.67:0/2431078656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c0754e0 0x7fc09c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 >> 172.21.15.67:0/2431078656 conn(0x7fc09c0fe660 msgr2=0x7fc09c111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 shutdown_connections 2024-09-10T23:13:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:13:55.959+0000 7fc09b200640 1 -- 172.21.15.67:0/2431078656 wait complete. 2024-09-10T23:13:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:56 smithi067 ceph-mon[30939]: pgmap v854: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-09-10T23:13:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:56 smithi145 ceph-mon[34397]: pgmap v854: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-09-10T23:13:56.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:56 smithi092 ceph-mon[34223]: pgmap v854: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-09-10T23:13:56.878 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b0b26892', quiesce took 4.4 seconds. Will release after: 18.35 2024-09-10T23:13:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:58 smithi145 ceph-mon[34397]: pgmap v855: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 MiB/s wr, 8 op/s 2024-09-10T23:13:58.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:58 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:58.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:58 smithi092 ceph-mon[34223]: pgmap v855: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 MiB/s wr, 8 op/s 2024-09-10T23:13:58.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:58 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:58.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:58 smithi067 ceph-mon[30939]: pgmap v855: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 MiB/s wr, 8 op/s 2024-09-10T23:13:58.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:58 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:59.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:13:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:13:59.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:59.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:59.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:13:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:13:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:59.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:13:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:14:00.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:00 smithi145 ceph-mon[34397]: pgmap v856: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.5 MiB/s wr, 6 op/s 2024-09-10T23:14:00.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:00 smithi092 ceph-mon[34223]: pgmap v856: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.5 MiB/s wr, 6 op/s 2024-09-10T23:14:00.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:00 smithi067 ceph-mon[30939]: pgmap v856: 129 pgs: 129 active+clean; 7.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.5 MiB/s wr, 6 op/s 2024-09-10T23:14:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:02 smithi145 ceph-mon[34397]: pgmap v857: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-09-10T23:14:02.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:02 smithi092 ceph-mon[34223]: pgmap v857: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-09-10T23:14:02.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:02 smithi067 ceph-mon[30939]: pgmap v857: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-09-10T23:14:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:04 smithi145 ceph-mon[34397]: pgmap v858: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-09-10T23:14:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:04 smithi145 ceph-mon[34397]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T23:14:04.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:04 smithi092 ceph-mon[34223]: pgmap v858: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-09-10T23:14:04.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:04 smithi092 ceph-mon[34223]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T23:14:04.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:04 smithi067 ceph-mon[30939]: pgmap v858: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-09-10T23:14:04.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:04 smithi067 ceph-mon[30939]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T23:14:05.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:05 smithi092 ceph-mon[34223]: pgmap v860: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 28 op/s 2024-09-10T23:14:05.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:05 smithi067 ceph-mon[30939]: pgmap v860: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 28 op/s 2024-09-10T23:14:05.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:05 smithi145 ceph-mon[34397]: pgmap v860: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 28 op/s 2024-09-10T23:14:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:07 smithi092 ceph-mon[34223]: pgmap v861: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.8 MiB/s wr, 29 op/s 2024-09-10T23:14:07.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:07 smithi067 ceph-mon[30939]: pgmap v861: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.8 MiB/s wr, 29 op/s 2024-09-10T23:14:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:07 smithi145 ceph-mon[34397]: pgmap v861: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.8 MiB/s wr, 29 op/s 2024-09-10T23:14:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:09 smithi145 ceph-mon[34397]: pgmap v862: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 28 op/s 2024-09-10T23:14:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:09 smithi145 ceph-mon[34397]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T23:14:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:09 smithi092 ceph-mon[34223]: pgmap v862: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 28 op/s 2024-09-10T23:14:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:09 smithi092 ceph-mon[34223]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T23:14:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:09 smithi067 ceph-mon[30939]: pgmap v862: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 28 op/s 2024-09-10T23:14:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:09 smithi067 ceph-mon[30939]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T23:14:11.850 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:11 smithi067 ceph-mon[30939]: pgmap v864: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 468 KiB/s wr, 11 op/s 2024-09-10T23:14:11.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:11 smithi145 ceph-mon[34397]: pgmap v864: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 468 KiB/s wr, 11 op/s 2024-09-10T23:14:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:11 smithi092 ceph-mon[34223]: pgmap v864: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 468 KiB/s wr, 11 op/s 2024-09-10T23:14:13.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:13 smithi145 ceph-mon[34397]: pgmap v865: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 404 KiB/s wr, 10 op/s 2024-09-10T23:14:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:13 smithi092 ceph-mon[34223]: pgmap v865: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 404 KiB/s wr, 10 op/s 2024-09-10T23:14:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:13 smithi067 ceph-mon[30939]: pgmap v865: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 404 KiB/s wr, 10 op/s 2024-09-10T23:14:15.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:14:15.618 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:15 smithi145 ceph-mon[34397]: pgmap v866: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-10T23:14:15.942 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:15 smithi067 ceph-mon[30939]: pgmap v866: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-10T23:14:16.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:15 smithi092 ceph-mon[34223]: pgmap v866: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-10T23:14:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.183+0000 7f1db0600640 1 -- 172.21.15.67:0/1060560983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8075e60 msgr2=0x7f1da81135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.183+0000 7f1db0600640 1 --2- 172.21.15.67:0/1060560983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8075e60 0x7f1da81135e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a410 comp rx=0 tx=0).stop 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 -- 172.21.15.67:0/1060560983 shutdown_connections 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 --2- 172.21.15.67:0/1060560983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8113b20 0x7f1da8115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 --2- 172.21.15.67:0/1060560983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8075e60 0x7f1da81135e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 --2- 172.21.15.67:0/1060560983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075540 0x7f1da8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 -- 172.21.15.67:0/1060560983 >> 172.21.15.67:0/1060560983 conn(0x7f1da80fe830 msgr2=0x7f1da8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 -- 172.21.15.67:0/1060560983 shutdown_connections 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.184+0000 7f1db0600640 1 -- 172.21.15.67:0/1060560983 wait complete. 2024-09-10T23:14:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 Processor -- start 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 -- start start 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8075540 0x7f1da819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 0x7f1da819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81192b0 con 0x7f1da8113b20 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8119130 con 0x7f1da8075540 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1db0600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8119430 con 0x7f1da8075e60 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1dae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1dae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58234/0 (socket says 172.21.15.67:58234) 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.185+0000 7f1dae600640 1 -- 172.21.15.67:0/2052001413 learned_addr learned my addr 172.21.15.67:0/2052001413 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dad200640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 0x7f1da819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dadc00640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8075540 0x7f1da819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 -- 172.21.15.67:0/2052001413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 msgr2=0x7f1da819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 0x7f1da819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 -- 172.21.15.67:0/2052001413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8075540 msgr2=0x7f1da819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8075540 0x7f1da819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 -- 172.21.15.67:0/2052001413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da81a5930 con 0x7f1da8113b20 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dad200640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 0x7f1da819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1dae600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1da000dac0 tx=0x7f1da000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0017070 con 0x7f1da8113b20 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da000be30 con 0x7f1da8113b20 2024-09-10T23:14:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0010460 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.186+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81a5c20 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.187+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81ac2e0 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.188+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da8075920 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.190+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1da0023060 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.191+0000 7f1da6000640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 0x7f1d900b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.191+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9519+0+0 (secure 0 0 0) 0x7f1da00e5ae0 con 0x7f1da8113b20 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.192+0000 7f1dadc00640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 0x7f1d900b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.192+0000 7f1dadc00640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 0x7f1d900b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009b60 tx=0x7f1d9c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.195+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1da00ac220 con 0x7f1da8113b20 2024-09-10T23:14:16.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.428+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1da81a09b0 con 0x7f1da8113b20 2024-09-10T23:14:16.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.428+0000 7f1da6000640 1 -- 172.21.15.67:0/2052001413 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f1da00abb20 con 0x7f1da8113b20 2024-09-10T23:14:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:14:16.431 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"btime":"2024-09-10T23:03:43:318772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:03:42.560848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:14:16.431 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-09-10T23:14:16.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 msgr2=0x7f1d900b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:16.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 0x7f1d900b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009b60 tx=0x7f1d9c005f90 comp rx=0 tx=0).stop 2024-09-10T23:14:16.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 msgr2=0x7f1da81a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:16.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1da000dac0 tx=0x7f1da000df80 comp rx=0 tx=0).stop 2024-09-10T23:14:16.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 shutdown_connections 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1d900b54d0 0x7f1d900b7990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1da8113b20 0x7f1da81a5160 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da8075e60 0x7f1da819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 --2- 172.21.15.67:0/2052001413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8075540 0x7f1da819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 >> 172.21.15.67:0/2052001413 conn(0x7f1da80fe830 msgr2=0x7f1da80ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:16.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 shutdown_connections 2024-09-10T23:14:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:16.431+0000 7f1db0600640 1 -- 172.21.15.67:0/2052001413 wait complete. 2024-09-10T23:14:16.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:16 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2052001413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:16.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:16 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2052001413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:17.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:16 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2052001413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:17 smithi145 ceph-mon[34397]: pgmap v867: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:17.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:17 smithi067 ceph-mon[30939]: pgmap v867: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:17.963 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id b0b26892 --release --await' 2024-09-10T23:14:17.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id b0b26892 --release --await 2024-09-10T23:14:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:17 smithi092 ceph-mon[34223]: pgmap v867: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:18.347 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 -- 172.21.15.67:0/940667420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8108075540 msgr2=0x7f8108075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 --2- 172.21.15.67:0/940667420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8108075540 0x7f8108075920 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009900 tx=0x7f80fc02a410 comp rx=0 tx=0).stop 2024-09-10T23:14:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 -- 172.21.15.67:0/940667420 shutdown_connections 2024-09-10T23:14:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 --2- 172.21.15.67:0/940667420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810810f3e0 0x7f81081117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 --2- 172.21.15.67:0/940667420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075f40 0x7f810810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 --2- 172.21.15.67:0/940667420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8108075540 0x7f8108075920 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 -- 172.21.15.67:0/940667420 >> 172.21.15.67:0/940667420 conn(0x7f81080fe830 msgr2=0x7f8108100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 -- 172.21.15.67:0/940667420 shutdown_connections 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.961+0000 7f810fa00640 1 -- 172.21.15.67:0/940667420 wait complete. 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.962+0000 7f810fa00640 1 Processor -- start 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.962+0000 7f810fa00640 1 -- start start 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075540 0x7f810819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8108075f40 0x7f810819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8108114e40 con 0x7f810810f3e0 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8108114cc0 con 0x7f8108075f40 2024-09-10T23:14:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810fa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8108114fc0 con 0x7f8108075540 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37876/0 (socket says 172.21.15.67:37876) 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 -- 172.21.15.67:0/3127294538 learned_addr learned my addr 172.21.15.67:0/3127294538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810d000640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075540 0x7f810819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f8107e00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8108075f40 0x7f810819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 -- 172.21.15.67:0/3127294538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075540 msgr2=0x7f810819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075540 0x7f810819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 -- 172.21.15.67:0/3127294538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8108075f40 msgr2=0x7f810819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8108075f40 0x7f810819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.963+0000 7f810da00640 1 -- 172.21.15.67:0/3127294538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81081a1500 con 0x7f810810f3e0 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f810da00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f810000eb40 tx=0x7f810000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f8105600640 1 -- 172.21.15.67:0/3127294538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100004b20 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f8105600640 1 -- 172.21.15.67:0/3127294538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8100004cc0 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f8105600640 1 -- 172.21.15.67:0/3127294538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100010690 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081a17f0 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.964+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a7eb0 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.967+0000 7f8112655740 1 Processor -- start 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.967+0000 7f8105600640 1 -- 172.21.15.67:0/3127294538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8100005540 con 0x7f810810f3e0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 -- start start 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53c9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559f53c11290 con 0x559f53ca68a0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f53c006c0 con 0x559f53c9a910 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8112655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f53c044d0 con 0x559f53c99ff0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f8107e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.968+0000 7f810d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47160/0 (socket says 172.21.15.67:47160) 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 -- 172.21.15.67:0/264106806 learned_addr learned my addr 172.21.15.67:0/264106806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f8107e00640 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53c9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 -- 172.21.15.67:0/264106806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 msgr2=0x559f53c9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53c9a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 -- 172.21.15.67:0/264106806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 msgr2=0x559f53ca8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 -- 172.21.15.67:0/264106806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f53b8d460 con 0x559f53c9a910 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f8107e00640 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53c9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f8105600640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 0x7f80f00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.969+0000 7f810d000640 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80fc000c00 tx=0x7f80fc031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed61e2b73552f387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f8105600640 1 -- 172.21.15.67:0/3127294538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9519+0+0 (secure 0 0 0) 0x7f81000e6c70 con 0x7f810810f3e0 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f80fa000640 1 -- 172.21.15.67:0/264106806 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc035030 con 0x559f53c9a910 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f8107e00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 0x7f80f00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f80fa000640 1 -- 172.21.15.67:0/264106806 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80fc040070 con 0x559f53c9a910 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f80fa000640 1 -- 172.21.15.67:0/264106806 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0037f0 con 0x559f53c9a910 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f8112655740 1 -- 172.21.15.67:0/264106806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 msgr2=0x559f53ca6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f8112655740 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80fc000c00 tx=0x7f80fc031ba0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.970+0000 7f8107e00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 0x7f80f00b7910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80f400d010 tx=0x7f80f4016040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 -- 172.21.15.67:0/264106806 shutdown_connections 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca6360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 --2- 172.21.15.67:0/264106806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53c9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 -- 172.21.15.67:0/264106806 >> 172.21.15.67:0/264106806 conn(0x559f53c97e20 msgr2=0x559f53c98fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 -- 172.21.15.67:0/264106806 shutdown_connections 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 -- 172.21.15.67:0/264106806 wait complete. 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 -- start start 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.971+0000 7f8112655740 1 Processor -- start 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c9a910 0x559f53ca3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559f53a61bb0 con 0x559f53ca68a0 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f53b9b6c0 con 0x559f53c99ff0 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f53ca94f0 con 0x559f53c9a910 2024-09-10T23:14:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8112655740 10 client.0 _ll_register_callbacks cb 0x559f53bb6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47174/0 (socket says 172.21.15.67:47174) 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 -- 172.21.15.67:0/3142852035 learned_addr learned my addr 172.21.15.67:0/3142852035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37894/0 (socket says 172.21.15.67:37894) 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f8107e00640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c9a910 0x559f53ca3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 -- 172.21.15.67:0/3142852035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c9a910 msgr2=0x559f53ca3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c9a910 0x559f53ca3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 -- 172.21.15.67:0/3142852035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 msgr2=0x559f53ca4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.972+0000 7f810d000640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f810d000640 1 -- 172.21.15.67:0/3142852035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5395eb00 con 0x559f53c99ff0 2024-09-10T23:14:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f810da00640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f810d000640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f80fc006010 tx=0x7f80fc004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1f52e2c8788d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f80eea00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc04c070 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9519+0+0 (secure 0 0 0) 0x7f80fc03c440 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc040070 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80fc058050 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f8112655740 10 client.35924 fetch_fsmap learned FSMap version 31 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.973+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f539d5960 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80fc040070 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f80fc051540 con 0x559f53c99ff0 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 10 client.35924 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 10 client.35924 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:14:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 20 client.35924 populate_metadata read hostname 'smithi067' 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53cb61c0 0x559f53cb85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 4 client.35924 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.974+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f53c131e0 con 0x559f53cb61c0 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.975+0000 7f8107e00640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53cb61c0 0x559f53cb85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.975+0000 7f8107e00640 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53cb61c0 0x559f53cb85b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da0c4c175299d21 server_cookie=2d48c2c5d2b450fc in_seq=0 out_seq=0 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.975+0000 7f80eea00640 10 client.35924 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.976+0000 7f80eea00640 1 -- 172.21.15.67:0/3142852035 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f53c131e0 con 0x559f53cb61c0 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.976+0000 7f80eea00640 10 client.35924 handle_command_reply: tid=0 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.976+0000 7f8112655740 1 client.35924 shutdown 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 msgr2=0x559f53ca37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f80fc006010 tx=0x7f80fc004600 comp rx=0 tx=0).stop 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 shutdown_connections 2024-09-10T23:14:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53cb61c0 0x559f53cb85b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53ca4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c9a910 0x559f53ca3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f80eea00640 0 client.35924 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 --2- 172.21.15.67:0/3142852035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c99ff0 0x559f53ca37d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 >> 172.21.15.67:0/3142852035 conn(0x559f53c97e20 msgr2=0x559f53c9d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 shutdown_connections 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 1 -- 172.21.15.67:0/3142852035 wait complete. 2024-09-10T23:14:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.977+0000 7f8112655740 20 client.35924 trim_cache size 0 max 16384 2024-09-10T23:14:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.995+0000 7f8112655740 1 Processor -- start 2024-09-10T23:14:18.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.995+0000 7f8112655740 1 -- start start 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 0x559f53ca0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b49f0 con 0x559f53c99ff0 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b22c0 con 0x559f53c9a910 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8112655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b0130 con 0x559f53ca68a0 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f810d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f810d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37904/0 (socket says 172.21.15.67:37904) 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f810d000640 1 -- 172.21.15.67:0/538320207 learned_addr learned my addr 172.21.15.67:0/538320207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f8107e00640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.996+0000 7f810da00640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 0x559f53ca0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 -- 172.21.15.67:0/538320207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 msgr2=0x559f53ca0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 0x559f53ca0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 -- 172.21.15.67:0/538320207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 msgr2=0x559f53ca0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 -- 172.21.15.67:0/538320207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f53c988a0 con 0x559f53c99ff0 2024-09-10T23:14:18.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810da00640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 0x559f53ca0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:14:19.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f810d000640 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0518b0 tx=0x7f80fc02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20cb3b997a737a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f80f9600640 1 -- 172.21.15.67:0/538320207 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc04c070 con 0x559f53c99ff0 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f80f9600640 1 -- 172.21.15.67:0/538320207 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80fc040040 con 0x559f53c99ff0 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.997+0000 7f80f9600640 1 -- 172.21.15.67:0/538320207 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc035050 con 0x559f53c99ff0 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- 172.21.15.67:0/538320207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 msgr2=0x559f53c9e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0518b0 tx=0x7f80fc02ab10 comp rx=0 tx=0).stop 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- 172.21.15.67:0/538320207 shutdown_connections 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53ca68a0 0x559f53ca0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53ca0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 --2- 172.21.15.67:0/538320207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53c99ff0 0x559f53c9e060 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- 172.21.15.67:0/538320207 >> 172.21.15.67:0/538320207 conn(0x559f53c97e20 msgr2=0x559f53ca5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- 172.21.15.67:0/538320207 shutdown_connections 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- 172.21.15.67:0/538320207 wait complete. 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.998+0000 7f8112655740 1 -- start start 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 Processor -- start 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53caed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53cad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b0780 con 0x559f53ca68a0 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b1bf0 con 0x559f53c9a910 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81081b0e50 con 0x559f53c99ff0 2024-09-10T23:14:19.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f8112655740 10 client.0 _ll_register_callbacks cb 0x7f80f400c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f810da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37906/0 (socket says 172.21.15.67:37906) 2024-09-10T23:14:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:18.999+0000 7f810da00640 1 -- 172.21.15.67:0/1452354728 learned_addr learned my addr 172.21.15.67:0/1452354728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:14:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f8107e00640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53cad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810d000640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53caed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 -- 172.21.15.67:0/1452354728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 msgr2=0x559f53caed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53caed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 -- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 msgr2=0x559f53cad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53cad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 -- 172.21.15.67:0/1452354728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f53cae2f0 con 0x559f53ca68a0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f810da00640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81000a9f20 tx=0x7f81000ad230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a435c6297bf53a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f80eea00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000ac7c0 con 0x559f53ca68a0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9519+0+0 (secure 0 0 0) 0x7f81000b4b10 con 0x559f53ca68a0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.000+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc040040 con 0x559f53ca68a0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8100098020 con 0x559f53ca68a0 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 10 client.16218 fetch_fsmap learned FSMap version 31 2024-09-10T23:14:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f53d23c20 con 0x559f53ca68a0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80fc040040 con 0x559f53ca68a0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f81000a8600 con 0x559f53ca68a0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 10 client.16218 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 10 client.16218 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 20 client.16218 populate_metadata read hostname 'smithi067' 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53d243a0 0x559f53d267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 4 client.16218 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b0b26892", "release": true, "await": true}] 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.001+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b0b26892", "release": true, "await": true}) -- 0x559f53c0ec90 con 0x559f53d243a0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.002+0000 7f810d000640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53d243a0 0x559f53d267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.002+0000 7f810d000640 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53d243a0 0x559f53d267c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f0d16a0ac66cc1 server_cookie=84035bed61052c76 in_seq=0 out_seq=0 2024-09-10T23:14:19.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.002+0000 7f80eea00640 10 client.16218 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:14:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.030+0000 7f80eea00640 1 -- 172.21.15.67:0/1452354728 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f53c0ec90 con 0x559f53d243a0 2024-09-10T23:14:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f80eea00640 10 client.16218 handle_command_reply: tid=0 2024-09-10T23:14:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 client.16218 shutdown 2024-09-10T23:14:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 msgr2=0x559f53cadd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81000a9f20 tx=0x7f81000ad230 comp rx=0 tx=0).stop 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 shutdown_connections 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x559f53d243a0 0x559f53d267c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559f53ca68a0 0x559f53cadd00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f53c9a910 0x559f53cad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 --2- 172.21.15.67:0/1452354728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f53c99ff0 0x559f53caed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 >> 172.21.15.67:0/1452354728 conn(0x559f53c97e20 msgr2=0x559f53ca4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f80eea00640 0 client.16218 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 shutdown_connections 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 1 -- 172.21.15.67:0/1452354728 wait complete. 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.031+0000 7f8112655740 20 client.16218 trim_cache size 0 max 16384 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 31, 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 48, 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "b0b26892": { 2024-09-10T23:14:19.034 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 48, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 142.8, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:14:19.035 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 msgr2=0x7f80f00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:19.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 0x7f80f00b7910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80f400d010 tx=0x7f80f4016040 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 msgr2=0x7f81081a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f810000eb40 tx=0x7f810000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 shutdown_connections 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f80f00b5450 0x7f80f00b7910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f810810f3e0 0x7f81081a0d30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8108075f40 0x7f810819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 --2- 172.21.15.67:0/3127294538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8108075540 0x7f810819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 >> 172.21.15.67:0/3127294538 conn(0x7f81080fe830 msgr2=0x7f81080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 shutdown_connections 2024-09-10T23:14:19.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:14:19.032+0000 7f810fa00640 1 -- 172.21.15.67:0/3127294538 wait complete. 2024-09-10T23:14:19.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:19 smithi067 ceph-mon[30939]: pgmap v868: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:19 smithi145 ceph-mon[34397]: pgmap v868: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:19.906 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b0b26892', total seconds elapsed: 27.4 2024-09-10T23:14:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:19 smithi092 ceph-mon[34223]: pgmap v868: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:14:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:21 smithi145 ceph-mon[34397]: pgmap v869: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 347 KiB/s wr, 7 op/s 2024-09-10T23:14:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:21 smithi092 ceph-mon[34223]: pgmap v869: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 347 KiB/s wr, 7 op/s 2024-09-10T23:14:22.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:21 smithi067 ceph-mon[30939]: pgmap v869: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 347 KiB/s wr, 7 op/s 2024-09-10T23:14:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:23 smithi145 ceph-mon[34397]: pgmap v870: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-09-10T23:14:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:23 smithi092 ceph-mon[34223]: pgmap v870: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-09-10T23:14:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:23 smithi067 ceph-mon[30939]: pgmap v870: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-09-10T23:14:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:25 smithi145 ceph-mon[34397]: pgmap v871: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.1 MiB/s wr, 7 op/s 2024-09-10T23:14:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:25 smithi092 ceph-mon[34223]: pgmap v871: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.1 MiB/s wr, 7 op/s 2024-09-10T23:14:26.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:25 smithi067 ceph-mon[30939]: pgmap v871: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.1 MiB/s wr, 7 op/s 2024-09-10T23:14:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:27 smithi145 ceph-mon[34397]: pgmap v872: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 9 op/s 2024-09-10T23:14:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:27 smithi092 ceph-mon[34223]: pgmap v872: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 9 op/s 2024-09-10T23:14:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:27 smithi067 ceph-mon[30939]: pgmap v872: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 MiB/s wr, 9 op/s 2024-09-10T23:14:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:29 smithi145 ceph-mon[34397]: pgmap v873: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.4 MiB/s wr, 7 op/s 2024-09-10T23:14:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:29 smithi092 ceph-mon[34223]: pgmap v873: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.4 MiB/s wr, 7 op/s 2024-09-10T23:14:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:29 smithi067 ceph-mon[30939]: pgmap v873: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.4 MiB/s wr, 7 op/s 2024-09-10T23:14:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:31 smithi145 ceph-mon[34397]: pgmap v874: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:14:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:31 smithi092 ceph-mon[34223]: pgmap v874: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:14:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:31 smithi067 ceph-mon[30939]: pgmap v874: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:14:33.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:33 smithi145 ceph-mon[34397]: pgmap v875: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:14:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:33 smithi092 ceph-mon[34223]: pgmap v875: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:14:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:33 smithi067 ceph-mon[30939]: pgmap v875: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:14:35.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:35 smithi145 ceph-mon[34397]: pgmap v876: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:14:36.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:35 smithi092 ceph-mon[34223]: pgmap v876: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:14:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:35 smithi067 ceph-mon[30939]: pgmap v876: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:14:37.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:37 smithi145 ceph-mon[34397]: pgmap v877: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:37 smithi092 ceph-mon[34223]: pgmap v877: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:37 smithi067 ceph-mon[30939]: pgmap v877: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:39 smithi092 ceph-mon[34223]: pgmap v878: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:14:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:39 smithi067 ceph-mon[30939]: pgmap v878: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:14:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:39 smithi145 ceph-mon[34397]: pgmap v878: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:14:41.932 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:41 smithi145 ceph-mon[34397]: pgmap v879: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:41 smithi092 ceph-mon[34223]: pgmap v879: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:42.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:41 smithi067 ceph-mon[30939]: pgmap v879: 129 pgs: 129 active+clean; 7.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:43 smithi092 ceph-mon[34223]: pgmap v880: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:14:44.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:43 smithi067 ceph-mon[30939]: pgmap v880: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:14:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:43 smithi145 ceph-mon[34397]: pgmap v880: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:14:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:45 smithi092 ceph-mon[34223]: pgmap v881: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:14:46.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:45 smithi067 ceph-mon[30939]: pgmap v881: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:14:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:45 smithi145 ceph-mon[34397]: pgmap v881: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:14:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:47 smithi092 ceph-mon[34223]: pgmap v882: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:14:48.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:47 smithi067 ceph-mon[30939]: pgmap v882: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:14:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:47 smithi145 ceph-mon[34397]: pgmap v882: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:14:49.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:48 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:49.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:48 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:49.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:48 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:49 smithi092 ceph-mon[34223]: pgmap v883: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:14:50.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:49 smithi067 ceph-mon[30939]: pgmap v883: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:14:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:49 smithi145 ceph-mon[34397]: pgmap v883: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:14:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:51 smithi092 ceph-mon[34223]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:52.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:51 smithi067 ceph-mon[30939]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:51 smithi145 ceph-mon[34397]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-10T23:14:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:53 smithi092 ceph-mon[34223]: pgmap v885: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:53 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:53 smithi067 ceph-mon[30939]: pgmap v885: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:53 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:53 smithi145 ceph-mon[34397]: pgmap v885: 129 pgs: 129 active+clean; 8.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:14:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:53 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:14:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:55 smithi092 ceph-mon[34223]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:14:56.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:55 smithi067 ceph-mon[30939]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:14:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:55 smithi145 ceph-mon[34397]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:14:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:57 smithi092 ceph-mon[34223]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:14:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:57 smithi067 ceph-mon[30939]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:14:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:57 smithi145 ceph-mon[34397]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-10T23:14:59.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:58 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:14:59.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:58 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:14:59.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:58 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:14:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:14:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:14:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:15:01.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:00 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:15:01.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:00 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:15:01.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:00 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:15:01.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:00 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:15:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:00 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:15:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:00 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:15:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:01 smithi092 ceph-mon[34223]: pgmap v889: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-09-10T23:15:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:01 smithi067 ceph-mon[30939]: pgmap v889: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-09-10T23:15:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:01 smithi145 ceph-mon[34397]: pgmap v889: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-09-10T23:15:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:03 smithi092 ceph-mon[34223]: pgmap v890: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 29 op/s 2024-09-10T23:15:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:03 smithi092 ceph-mon[34223]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T23:15:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:03 smithi067 ceph-mon[30939]: pgmap v890: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 29 op/s 2024-09-10T23:15:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:03 smithi067 ceph-mon[30939]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T23:15:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:03 smithi145 ceph-mon[34397]: pgmap v890: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 29 op/s 2024-09-10T23:15:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:03 smithi145 ceph-mon[34397]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T23:15:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:05 smithi092 ceph-mon[34223]: pgmap v892: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:05 smithi067 ceph-mon[30939]: pgmap v892: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:05 smithi145 ceph-mon[34397]: pgmap v892: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:07 smithi092 ceph-mon[34223]: pgmap v893: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:08.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:07 smithi067 ceph-mon[30939]: pgmap v893: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:07 smithi145 ceph-mon[34397]: pgmap v893: 129 pgs: 10 active+clean+snaptrim_wait, 119 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:15:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:09 smithi092 ceph-mon[34223]: pgmap v894: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 35 op/s 2024-09-10T23:15:10.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:09 smithi067 ceph-mon[30939]: pgmap v894: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 35 op/s 2024-09-10T23:15:10.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:09 smithi145 ceph-mon[34397]: pgmap v894: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 35 op/s 2024-09-10T23:15:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:11 smithi092 ceph-mon[34223]: pgmap v895: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 35 op/s 2024-09-10T23:15:12.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:11 smithi067 ceph-mon[30939]: pgmap v895: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 35 op/s 2024-09-10T23:15:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:11 smithi145 ceph-mon[34397]: pgmap v895: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 35 op/s 2024-09-10T23:15:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:13 smithi092 ceph-mon[34223]: pgmap v896: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-10T23:15:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:13 smithi092 ceph-mon[34223]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T23:15:14.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:13 smithi067 ceph-mon[30939]: pgmap v896: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-10T23:15:14.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:13 smithi067 ceph-mon[30939]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T23:15:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:13 smithi145 ceph-mon[34397]: pgmap v896: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-10T23:15:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:13 smithi145 ceph-mon[34397]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T23:15:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:15 smithi092 ceph-mon[34223]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:15:16.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:15 smithi067 ceph-mon[30939]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:15:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:15 smithi145 ceph-mon[34397]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:15:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:17 smithi092 ceph-mon[34223]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:18.321 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:17 smithi145 ceph-mon[34397]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:18.321 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:17 smithi067 ceph-mon[30939]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:19 smithi092 ceph-mon[34223]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:15:20.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:19 smithi067 ceph-mon[30939]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:15:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:19 smithi145 ceph-mon[34397]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:15:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:21 smithi092 ceph-mon[34223]: pgmap v901: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:21 smithi067 ceph-mon[30939]: pgmap v901: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:21 smithi145 ceph-mon[34397]: pgmap v901: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:23 smithi092 ceph-mon[34223]: pgmap v902: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:23 smithi067 ceph-mon[30939]: pgmap v902: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:23 smithi145 ceph-mon[34397]: pgmap v902: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:15:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:25 smithi092 ceph-mon[34223]: pgmap v903: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:15:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:25 smithi067 ceph-mon[30939]: pgmap v903: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:15:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:25 smithi145 ceph-mon[34397]: pgmap v903: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 14 op/s 2024-09-10T23:15:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:27 smithi092 ceph-mon[34223]: pgmap v904: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:27 smithi067 ceph-mon[30939]: pgmap v904: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:27 smithi145 ceph-mon[34397]: pgmap v904: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:29 smithi092 ceph-mon[34223]: pgmap v905: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:15:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:29 smithi067 ceph-mon[30939]: pgmap v905: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:15:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:29 smithi145 ceph-mon[34397]: pgmap v905: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:15:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:31 smithi092 ceph-mon[34223]: pgmap v906: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-09-10T23:15:32.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:31 smithi067 ceph-mon[30939]: pgmap v906: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-09-10T23:15:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:31 smithi145 ceph-mon[34397]: pgmap v906: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-09-10T23:15:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:33 smithi092 ceph-mon[34223]: pgmap v907: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:34.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:33 smithi067 ceph-mon[30939]: pgmap v907: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:34.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:33 smithi145 ceph-mon[34397]: pgmap v907: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:15:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:35 smithi092 ceph-mon[34223]: pgmap v908: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:15:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:35 smithi067 ceph-mon[30939]: pgmap v908: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:15:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:35 smithi145 ceph-mon[34397]: pgmap v908: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:15:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:37 smithi092 ceph-mon[34223]: pgmap v909: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:37 smithi067 ceph-mon[30939]: pgmap v909: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:37 smithi145 ceph-mon[34397]: pgmap v909: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:40.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:39 smithi092 ceph-mon[34223]: pgmap v910: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:15:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:39 smithi067 ceph-mon[30939]: pgmap v910: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:15:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:39 smithi145 ceph-mon[34397]: pgmap v910: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:15:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:41 smithi092 ceph-mon[34223]: pgmap v911: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:41 smithi067 ceph-mon[30939]: pgmap v911: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:41 smithi145 ceph-mon[34397]: pgmap v911: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:15:43.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:43 smithi067 ceph-mon[30939]: pgmap v912: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:15:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:43 smithi092 ceph-mon[34223]: pgmap v912: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:15:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:43 smithi145 ceph-mon[34397]: pgmap v912: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:15:44.407 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.8 2024-09-10T23:15:44.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:15:44.839 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 -- 172.21.15.67:0/2991182382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc10075540 msgr2=0x7fbc10075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 --2- 172.21.15.67:0/2991182382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc10075540 0x7fbc10075920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002a410 comp rx=0 tx=0).stop 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 -- 172.21.15.67:0/2991182382 shutdown_connections 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 --2- 172.21.15.67:0/2991182382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc1010f220 0x7fbc10111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 --2- 172.21.15.67:0/2991182382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075f40 0x7fbc1010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 --2- 172.21.15.67:0/2991182382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc10075540 0x7fbc10075920 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 -- 172.21.15.67:0/2991182382 >> 172.21.15.67:0/2991182382 conn(0x7fbc100fe6d0 msgr2=0x7fbc10100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 -- 172.21.15.67:0/2991182382 shutdown_connections 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.548+0000 7fbc14a00640 1 -- 172.21.15.67:0/2991182382 wait complete. 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.549+0000 7fbc14a00640 1 Processor -- start 2024-09-10T23:15:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.549+0000 7fbc14a00640 1 -- start start 2024-09-10T23:15:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 0x7fbc10102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 0x7fbc10102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 0x7fbc10103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10114c80 con 0x7fbc1010f220 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10114b00 con 0x7fbc10075540 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc14a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10114e00 con 0x7fbc10075f40 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 0x7fbc10103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc07e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 0x7fbc10102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc0f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 0x7fbc10102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 0x7fbc10103380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50234/0 (socket says 172.21.15.67:50234) 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc07e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 0x7fbc10102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51214/0 (socket says 172.21.15.67:51214) 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.550+0000 7fbc0fe00640 1 -- 172.21.15.67:0/3184832476 learned_addr learned my addr 172.21.15.67:0/3184832476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 -- 172.21.15.67:0/3184832476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 msgr2=0x7fbc10102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 0x7fbc10102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 -- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 msgr2=0x7fbc10103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 0x7fbc10103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 -- 172.21.15.67:0/3184832476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc101ab720 con 0x7fbc10075540 2024-09-10T23:15:45.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0f400640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 0x7fbc10102900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc00004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:45.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0003f070 con 0x7fbc10075540 2024-09-10T23:15:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc00003a80 con 0x7fbc10075540 2024-09-10T23:15:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00035070 con 0x7fbc10075540 2024-09-10T23:15:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.551+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc101ab9b0 con 0x7fbc10075540 2024-09-10T23:15:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.552+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101abe10 con 0x7fbc10075540 2024-09-10T23:15:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.553+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc10075920 con 0x7fbc10075540 2024-09-10T23:15:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.556+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbc0004b060 con 0x7fbc10075540 2024-09-10T23:15:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.557+0000 7fbc0cc00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 0x7fbbf40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.557+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9519+0+0 (secure 0 0 0) 0x7fbc00106870 con 0x7fbc10075540 2024-09-10T23:15:45.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.557+0000 7fbc07e00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 0x7fbbf40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.558+0000 7fbc07e00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 0x7fbbf40b79c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc10106410 tx=0x7fbbf8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:45.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.559+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbc000cc960 con 0x7fbc10075540 2024-09-10T23:15:45.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:45 smithi067 ceph-mon[30939]: pgmap v913: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:15:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.836+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc101ac260 con 0x7fbc10075540 2024-09-10T23:15:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.837+0000 7fbc0cc00640 1 -- 172.21.15.67:0/3184832476 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11906 (secure 0 0 0) 0x7fbc000d1390 con 0x7fbc10075540 2024-09-10T23:15:45.839 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:15:45.839 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"btime":"2024-09-10T23:14:52:659129+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:14:52.238931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:15:45.839 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 33 2024-09-10T23:15:45.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 msgr2=0x7fbbf40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 0x7fbbf40b79c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc10106410 tx=0x7fbbf8005f90 comp rx=0 tx=0).stop 2024-09-10T23:15:45.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 msgr2=0x7fbc10102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 0x7fbc10102900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc00004040 comp rx=0 tx=0).stop 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 shutdown_connections 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbbf40b5500 0x7fbbf40b79c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1010f220 0x7fbc10103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc10075f40 0x7fbc10102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 --2- 172.21.15.67:0/3184832476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10075540 0x7fbc10102900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 >> 172.21.15.67:0/3184832476 conn(0x7fbc100fe6d0 msgr2=0x7fbc101113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 shutdown_connections 2024-09-10T23:15:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:45.840+0000 7fbc14a00640 1 -- 172.21.15.67:0/3184832476 wait complete. 2024-09-10T23:15:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:45 smithi092 ceph-mon[34223]: pgmap v913: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:15:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:45 smithi145 ceph-mon[34397]: pgmap v913: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:15:47.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:46 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3184832476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:47.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:46 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3184832476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:47.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:46 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3184832476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:47.391 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0' 2024-09-10T23:15:47.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0 2024-09-10T23:15:47.817 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:47 smithi067 ceph-mon[30939]: pgmap v914: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:47 smithi145 ceph-mon[34397]: pgmap v914: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:47 smithi092 ceph-mon[34223]: pgmap v914: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 -- 172.21.15.67:0/1801071309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec102c10 msgr2=0x7fe1ec102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 --2- 172.21.15.67:0/1801071309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec102c10 0x7fe1ec102ff0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e002a410 comp rx=0 tx=0).stop 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 -- 172.21.15.67:0/1801071309 shutdown_connections 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 --2- 172.21.15.67:0/1801071309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 --2- 172.21.15.67:0/1801071309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 0x7fe1ec108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 --2- 172.21.15.67:0/1801071309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec102c10 0x7fe1ec102ff0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 -- 172.21.15.67:0/1801071309 >> 172.21.15.67:0/1801071309 conn(0x7fe1ec0fe660 msgr2=0x7fe1ec100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 -- 172.21.15.67:0/1801071309 shutdown_connections 2024-09-10T23:15:48.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.582+0000 7fe1f3200640 1 -- 172.21.15.67:0/1801071309 wait complete. 2024-09-10T23:15:48.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.583+0000 7fe1f3200640 1 Processor -- start 2024-09-10T23:15:48.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 -- start start 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec102c10 0x7fe1ec199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 0x7fe1ec199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112b10 con 0x7fe1ec108fa0 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112990 con 0x7fe1ec102c10 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112c90 con 0x7fe1ec108620 2024-09-10T23:15:48.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50256/0 (socket says 172.21.15.67:50256) 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.584+0000 7fe1f2800640 1 -- 172.21.15.67:0/2459904206 learned_addr learned my addr 172.21.15.67:0/2459904206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec102c10 0x7fe1ec199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 0x7fe1ec199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 -- 172.21.15.67:0/2459904206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 msgr2=0x7fe1ec199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 0x7fe1ec199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 -- 172.21.15.67:0/2459904206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec102c10 msgr2=0x7fe1ec199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec102c10 0x7fe1ec199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 -- 172.21.15.67:0/2459904206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec19f0b0 con 0x7fe1ec108fa0 2024-09-10T23:15:48.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.585+0000 7fe1f2800640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800eaa0 tx=0x7fe1e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.586+0000 7fe1e6a00640 1 -- 172.21.15.67:0/2459904206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800cdb0 con 0x7fe1ec108fa0 2024-09-10T23:15:48.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.586+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec19f3a0 con 0x7fe1ec108fa0 2024-09-10T23:15:48.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.586+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec075380 con 0x7fe1ec108fa0 2024-09-10T23:15:48.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.586+0000 7fe1e6a00640 1 -- 172.21.15.67:0/2459904206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1e8005020 con 0x7fe1ec108fa0 2024-09-10T23:15:48.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.586+0000 7fe1e6a00640 1 -- 172.21.15.67:0/2459904206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e801f060 con 0x7fe1ec108fa0 2024-09-10T23:15:48.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.590+0000 7fe1f7470740 1 Processor -- start 2024-09-10T23:15:48.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.591+0000 7fe1e6a00640 1 -- 172.21.15.67:0/2459904206 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe1e800f040 con 0x7fe1ec108fa0 2024-09-10T23:15:48.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.591+0000 7fe1f7470740 1 -- start start 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad52f0 0x56497fad56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497fae3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56497fa4aca0 con 0x56497fae1ba0 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56497fa3a0d0 con 0x56497fad5c10 2024-09-10T23:15:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f7470740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56497fa3dee0 con 0x56497fad52f0 2024-09-10T23:15:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad52f0 0x56497fad56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59288/0 (socket says 172.21.15.67:59288) 2024-09-10T23:15:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f1e00640 1 -- 172.21.15.67:0/3054941399 learned_addr learned my addr 172.21.15.67:0/3054941399 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.592+0000 7fe1f1400640 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497fae3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 -- 172.21.15.67:0/3054941399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad52f0 msgr2=0x56497fad56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad52f0 0x56497fad56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 -- 172.21.15.67:0/3054941399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 msgr2=0x56497fae3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497fae3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 -- 172.21.15.67:0/3054941399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497f9ec400 con 0x56497fad5c10 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1e6a00640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 0x7fe1d40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0005ec0 tx=0x7fe1e0031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6eae6f9f41c7144 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.593+0000 7fe1e6a00640 1 -- 172.21.15.67:0/2459904206 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9519+0+0 (secure 0 0 0) 0x7fe1e80e5f60 con 0x7fe1ec108fa0 2024-09-10T23:15:48.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 0x7fe1d40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1db000640 1 -- 172.21.15.67:0/3054941399 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0035030 con 0x56497fad5c10 2024-09-10T23:15:48.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1db000640 1 -- 172.21.15.67:0/3054941399 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1e0040070 con 0x56497fad5c10 2024-09-10T23:15:48.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1db000640 1 -- 172.21.15.67:0/3054941399 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0003850 con 0x56497fad5c10 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 0x7fe1d40b7a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00bfd0 tx=0x7fe1dc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1f7470740 1 -- 172.21.15.67:0/3054941399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 msgr2=0x56497fae1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.594+0000 7fe1f7470740 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0005ec0 tx=0x7fe1e0031ba0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 -- 172.21.15.67:0/3054941399 shutdown_connections 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497fae3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae1660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 --2- 172.21.15.67:0/3054941399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad52f0 0x56497fad56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 -- 172.21.15.67:0/3054941399 >> 172.21.15.67:0/3054941399 conn(0x56497fad3120 msgr2=0x56497fad42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 -- 172.21.15.67:0/3054941399 shutdown_connections 2024-09-10T23:15:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 -- 172.21.15.67:0/3054941399 wait complete. 2024-09-10T23:15:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.595+0000 7fe1f7470740 1 -- start start 2024-09-10T23:15:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 Processor -- start 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 0x56497fae13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497fade830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497faded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56497f89b7a0 con 0x56497fae1ba0 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56497f9dfa10 con 0x56497fad52f0 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56497fae47c0 con 0x56497fad5c10 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.596+0000 7fe1f7470740 10 client.0 _ll_register_callbacks cb 0x56497f9efbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497fade830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 0x56497fae13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497fade830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51240/0 (socket says 172.21.15.67:51240) 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1400640 1 -- 172.21.15.67:0/1217902229 learned_addr learned my addr 172.21.15.67:0/1217902229 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f2800640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497faded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217902229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 msgr2=0x56497fade830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497fade830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217902229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 msgr2=0x56497faded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497faded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.597+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217902229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497f7a7b90 con 0x56497fad52f0 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.598+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 0x56497fae13e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00057c0 tx=0x7fe1e0031610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f380b2763e8a05aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.598+0000 7fe1cf400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:15:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.598+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0035070 con 0x56497fad52f0 2024-09-10T23:15:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.598+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9519+0+0 (secure 0 0 0) 0x7fe1e0004240 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.598+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e0040070 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1e0009520 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 10 client.35948 fetch_fsmap learned FSMap version 33 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56497faf00e0 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1e0040070 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e004d460 con 0x56497fad52f0 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 10 client.35948 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 10 client.35948 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 20 client.35948 populate_metadata read hostname 'smithi067' 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497faf1430 0x56497faf3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 4 client.35948 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.599+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56497fa4cbf0 con 0x56497faf1430 2024-09-10T23:15:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.600+0000 7fe1f1400640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497faf1430 0x56497faf3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.601+0000 7fe1f1400640 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497faf1430 0x56497faf3820 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59768b69859cab5c server_cookie=31cb2b03dbbe3188 in_seq=0 out_seq=0 2024-09-10T23:15:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.601+0000 7fe1cf400640 10 client.35948 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:15:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1cf400640 1 -- 172.21.15.67:0/1217902229 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56497fa4cbf0 con 0x56497faf1430 2024-09-10T23:15:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1cf400640 10 client.35948 handle_command_reply: tid=0 2024-09-10T23:15:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 client.35948 shutdown 2024-09-10T23:15:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 msgr2=0x56497fae13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 0x56497fae13e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00057c0 tx=0x7fe1e0031610 comp rx=0 tx=0).stop 2024-09-10T23:15:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 shutdown_connections 2024-09-10T23:15:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497faf1430 0x56497faf3820 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fae1ba0 0x56497faded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497fade830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217902229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad52f0 0x56497fae13e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 >> 172.21.15.67:0/1217902229 conn(0x56497fad3120 msgr2=0x56497fad8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1cf400640 0 client.35948 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.602+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 shutdown_connections 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.603+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217902229 wait complete. 2024-09-10T23:15:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.603+0000 7fe1f7470740 20 client.35948 trim_cache size 0 max 16384 2024-09-10T23:15:48.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.626+0000 7fe1f7470740 1 Processor -- start 2024-09-10T23:15:48.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.626+0000 7fe1f7470740 1 -- start start 2024-09-10T23:15:48.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.626+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.626+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fae1ba0 0x56497fae34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f7470740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0012a40 con 0x56497fad52f0 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f7470740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00102b0 con 0x56497fad5c10 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f7470740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d000e120 con 0x56497fae1ba0 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50286/0 (socket says 172.21.15.67:50286) 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217880145 learned_addr learned my addr 172.21.15.67:0/1217880145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1400640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f2800640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fae1ba0 0x56497fae34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217880145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fae1ba0 msgr2=0x56497fae34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fae1ba0 0x56497fae34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217880145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 msgr2=0x56497fae2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1e00640 1 -- 172.21.15.67:0/1217880145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497fad44e0 con 0x56497fad52f0 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.627+0000 7fe1f1400640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.628+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e002abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f894a4d5eb68b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.628+0000 7fe1da600640 1 -- 172.21.15.67:0/1217880145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e005c030 con 0x56497fad52f0 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.628+0000 7fe1da600640 1 -- 172.21.15.67:0/1217880145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1e0040070 con 0x56497fad52f0 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.628+0000 7fe1da600640 1 -- 172.21.15.67:0/1217880145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0035040 con 0x56497fad52f0 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217880145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 msgr2=0x56497fad2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e002abc0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217880145 shutdown_connections 2024-09-10T23:15:48.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fae1ba0 0x56497fae34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fad5c10 0x56497fae2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 --2- 172.21.15.67:0/1217880145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497fad2af0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217880145 >> 172.21.15.67:0/1217880145 conn(0x56497fad3120 msgr2=0x56497faddd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217880145 shutdown_connections 2024-09-10T23:15:48.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.629+0000 7fe1f7470740 1 -- 172.21.15.67:0/1217880145 wait complete. 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 -- start start 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 Processor -- start 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497faf7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fae1ba0 0x56497faf3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d000e770 con 0x56497fad52f0 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d000fbe0 con 0x56497fae1ba0 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d000ee40 con 0x56497fad5c10 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.630+0000 7fe1f7470740 10 client.0 _ll_register_callbacks cb 0x7fe1dc00be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:48.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50298/0 (socket says 172.21.15.67:50298) 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 -- 172.21.15.67:0/2919017658 learned_addr learned my addr 172.21.15.67:0/2919017658 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f2800640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fae1ba0 0x56497faf3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497faf7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 -- 172.21.15.67:0/2919017658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 msgr2=0x56497faf7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497faf7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 -- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fae1ba0 msgr2=0x56497faf3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fae1ba0 0x56497faf3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1e00640 1 -- 172.21.15.67:0/2919017658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497faf3c30 con 0x56497fad52f0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.631+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497faf7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1f1e00640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0004720 tx=0x7fe1e004d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d151e7116aee26c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1cf400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:15:48.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e005c070 con 0x56497fad52f0 2024-09-10T23:15:48.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9519+0+0 (secure 0 0 0) 0x7fe1e004d9d0 con 0x56497fad52f0 2024-09-10T23:15:48.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e0040070 con 0x56497fad52f0 2024-09-10T23:15:48.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1e006a050 con 0x56497fad52f0 2024-09-10T23:15:48.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1f7470740 10 client.16242 fetch_fsmap learned FSMap version 33 2024-09-10T23:15:48.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.632+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56497fb5ecd0 con 0x56497fad52f0 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1e0040070 con 0x56497fad52f0 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e0061460 con 0x56497fad52f0 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 10 client.16242 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 10 client.16242 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 20 client.16242 populate_metadata read hostname 'smithi067' 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497fb5f220 0x56497fb61640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 4 client.16242 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}] 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.633+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}) -- 0x56497fa486a0 con 0x56497fb5f220 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.634+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497fb5f220 0x56497fb61640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:48.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.634+0000 7fe1f1400640 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497fb5f220 0x56497fb61640 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71722ff0b98ba9a1 server_cookie=6f68b18ac71f2db4 in_seq=0 out_seq=0 2024-09-10T23:15:48.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.634+0000 7fe1cf400640 10 client.16242 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:15:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.790+0000 7fe1cf400640 1 -- 172.21.15.67:0/2919017658 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56497fa486a0 con 0x56497fb5f220 2024-09-10T23:15:48.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.790+0000 7fe1cf400640 10 client.16242 handle_command_reply: tid=0 2024-09-10T23:15:48.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 client.16242 shutdown 2024-09-10T23:15:48.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 msgr2=0x56497faf6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0004720 tx=0x7fe1e004d460 comp rx=0 tx=0).stop 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 shutdown_connections 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56497fb5f220 0x56497fb61640 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56497fae1ba0 0x56497faf3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56497fad5c10 0x56497faf7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 --2- 172.21.15.67:0/2919017658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56497fad52f0 0x56497faf6cf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 >> 172.21.15.67:0/2919017658 conn(0x56497fad3120 msgr2=0x56497fadfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1cf400640 0 client.16242 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 shutdown_connections 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 1 -- 172.21.15.67:0/2919017658 wait complete. 2024-09-10T23:15:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.791+0000 7fe1f7470740 20 client.16242 trim_cache size 0 max 16384 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 33, 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 50, 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:15:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "54e8cfa9": { 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 50, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 154.8, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:15:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:15:48.820 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 msgr2=0x7fe1d40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 0x7fe1d40b7a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00bfd0 tx=0x7fe1dc00b390 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 msgr2=0x7fe1ec19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800eaa0 tx=0x7fe1e800ef60 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 shutdown_connections 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe1d40b5580 0x7fe1d40b7a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ec108fa0 0x7fe1ec19e8e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1ec108620 0x7fe1ec199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 --2- 172.21.15.67:0/2459904206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1ec102c10 0x7fe1ec199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 >> 172.21.15.67:0/2459904206 conn(0x7fe1ec0fe660 msgr2=0x7fe1ec106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.792+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 shutdown_connections 2024-09-10T23:15:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:15:48.793+0000 7fe1f3200640 1 -- 172.21.15.67:0/2459904206 wait complete. 2024-09-10T23:15:49.762 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '54e8cfa9', quiesce took 5.4 seconds. Will release after: 29.4 2024-09-10T23:15:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:49 smithi067 ceph-mon[30939]: pgmap v915: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:49 smithi145 ceph-mon[34397]: pgmap v915: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:49 smithi092 ceph-mon[34223]: pgmap v915: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:51 smithi067 ceph-mon[30939]: pgmap v916: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-09-10T23:15:52.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:51 smithi092 ceph-mon[34223]: pgmap v916: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-09-10T23:15:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:51 smithi145 ceph-mon[34397]: pgmap v916: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-09-10T23:15:54.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:53 smithi067 ceph-mon[30939]: pgmap v917: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:54.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:53 smithi145 ceph-mon[34397]: pgmap v917: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:53 smithi092 ceph-mon[34223]: pgmap v917: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:15:56.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:55 smithi067 ceph-mon[30939]: pgmap v918: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:15:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:55 smithi145 ceph-mon[34397]: pgmap v918: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:15:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:55 smithi092 ceph-mon[34223]: pgmap v918: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-10T23:15:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:57 smithi067 ceph-mon[30939]: pgmap v919: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 13 op/s 2024-09-10T23:15:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:57 smithi145 ceph-mon[34397]: pgmap v919: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 13 op/s 2024-09-10T23:15:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:57 smithi092 ceph-mon[34223]: pgmap v919: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 13 op/s 2024-09-10T23:16:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:59 smithi067 ceph-mon[30939]: pgmap v920: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 9 op/s 2024-09-10T23:16:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:15:59 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:59 smithi145 ceph-mon[34397]: pgmap v920: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 9 op/s 2024-09-10T23:16:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:15:59 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:59 smithi092 ceph-mon[34223]: pgmap v920: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 9 op/s 2024-09-10T23:16:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:15:59 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:16:01.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:16:01.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:16:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:16:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:16:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:16:01.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:16:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:01 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:16:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:01 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:16:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:01 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:16:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:01 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:16:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:01 smithi145 ceph-mon[34397]: pgmap v921: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 11 op/s 2024-09-10T23:16:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:01 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:16:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:01 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:16:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:01 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:16:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:01 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:16:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:01 smithi092 ceph-mon[34223]: pgmap v921: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 11 op/s 2024-09-10T23:16:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:01 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:16:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:01 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:16:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:01 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:16:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:01 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:16:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:01 smithi067 ceph-mon[30939]: pgmap v921: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 11 op/s 2024-09-10T23:16:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:03 smithi145 ceph-mon[34397]: pgmap v922: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 850 KiB/s wr, 21 op/s 2024-09-10T23:16:04.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:03 smithi092 ceph-mon[34223]: pgmap v922: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 850 KiB/s wr, 21 op/s 2024-09-10T23:16:04.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:03 smithi067 ceph-mon[30939]: pgmap v922: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 850 KiB/s wr, 21 op/s 2024-09-10T23:16:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:05 smithi145 ceph-mon[34397]: pgmap v923: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 531 KiB/s wr, 23 op/s 2024-09-10T23:16:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:05 smithi092 ceph-mon[34223]: pgmap v923: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 531 KiB/s wr, 23 op/s 2024-09-10T23:16:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:05 smithi067 ceph-mon[30939]: pgmap v923: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 531 KiB/s wr, 23 op/s 2024-09-10T23:16:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:07 smithi145 ceph-mon[34397]: pgmap v924: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 191 KiB/s wr, 25 op/s 2024-09-10T23:16:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:07 smithi092 ceph-mon[34223]: pgmap v924: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 191 KiB/s wr, 25 op/s 2024-09-10T23:16:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:07 smithi067 ceph-mon[30939]: pgmap v924: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 191 KiB/s wr, 25 op/s 2024-09-10T23:16:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:08 smithi145 ceph-mon[34397]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T23:16:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:08 smithi092 ceph-mon[34223]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T23:16:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:08 smithi067 ceph-mon[30939]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T23:16:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:09 smithi145 ceph-mon[34397]: pgmap v926: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 229 KiB/s wr, 27 op/s 2024-09-10T23:16:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:09 smithi092 ceph-mon[34223]: pgmap v926: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 229 KiB/s wr, 27 op/s 2024-09-10T23:16:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:09 smithi067 ceph-mon[30939]: pgmap v926: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 229 KiB/s wr, 27 op/s 2024-09-10T23:16:12.100 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:11 smithi067 ceph-mon[30939]: pgmap v927: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:16:12.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:11 smithi145 ceph-mon[34397]: pgmap v927: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:16:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:11 smithi092 ceph-mon[34223]: pgmap v927: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:16:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:13 smithi067 ceph-mon[30939]: pgmap v928: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-10T23:16:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:13 smithi145 ceph-mon[34397]: pgmap v928: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-10T23:16:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:13 smithi092 ceph-mon[34223]: pgmap v928: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-09-10T23:16:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:15 smithi145 ceph-mon[34397]: pgmap v929: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T23:16:16.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:15 smithi092 ceph-mon[34223]: pgmap v929: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T23:16:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:15 smithi067 ceph-mon[30939]: pgmap v929: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T23:16:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:17 smithi145 ceph-mon[34397]: pgmap v930: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:16:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:17 smithi092 ceph-mon[34223]: pgmap v930: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:16:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:17 smithi067 ceph-mon[30939]: pgmap v930: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-10T23:16:19.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:16:19.556 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:19.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:19 smithi067 ceph-mon[30939]: pgmap v931: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 290 B/s wr, 6 op/s 2024-09-10T23:16:20.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:19 smithi067 ceph-mon[30939]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T23:16:20.165 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:19 smithi092 ceph-mon[34223]: pgmap v931: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 290 B/s wr, 6 op/s 2024-09-10T23:16:20.165 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:19 smithi092 ceph-mon[34223]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T23:16:20.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:19 smithi145 ceph-mon[34397]: pgmap v931: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 290 B/s wr, 6 op/s 2024-09-10T23:16:20.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:19 smithi145 ceph-mon[34397]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 -- 172.21.15.67:0/2453286311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b81087d0 msgr2=0x7fb3b8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 --2- 172.21.15.67:0/2453286311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b81087d0 0x7fb3b8108c10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009900 tx=0x7fb3a402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 -- 172.21.15.67:0/2453286311 shutdown_connections 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 --2- 172.21.15.67:0/2453286311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b8109150 0x7fb3b810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 --2- 172.21.15.67:0/2453286311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b81087d0 0x7fb3b8108c10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 --2- 172.21.15.67:0/2453286311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 -- 172.21.15.67:0/2453286311 >> 172.21.15.67:0/2453286311 conn(0x7fb3b80fe830 msgr2=0x7fb3b8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 -- 172.21.15.67:0/2453286311 shutdown_connections 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.141+0000 7fb3be400640 1 -- 172.21.15.67:0/2453286311 wait complete. 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.142+0000 7fb3be400640 1 Processor -- start 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.142+0000 7fb3be400640 1 -- start start 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b81087d0 0x7fb3b80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b8109150 0x7fb3b8075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8112c80 con 0x7fb3b8109150 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8112b00 con 0x7fb3b8102dc0 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3be400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8112e00 con 0x7fb3b81087d0 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3b6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b81087d0 0x7fb3b80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:37850/0 (socket says 172.21.15.67:37850) 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3b7400640 1 -- 172.21.15.67:0/1769770491 learned_addr learned my addr 172.21.15.67:0/1769770491 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.143+0000 7fb3b7e00640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b8109150 0x7fb3b8075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 -- 172.21.15.67:0/1769770491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b81087d0 msgr2=0x7fb3b80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b81087d0 0x7fb3b80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 -- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b8109150 msgr2=0x7fb3b8075ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b8109150 0x7fb3b8075ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 -- 172.21.15.67:0/1769770491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b81abaf0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.144+0000 7fb3b7400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800d950 tx=0x7fb3a800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.145+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8017030 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.145+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b81abcc0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.145+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3a8004020 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.145+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80055d0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.145+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b81ac1c0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.146+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b81031a0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.149+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb3a8009a30 con 0x7fb3b8102dc0 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.150+0000 7fb3afe00640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 0x7fb39c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.150+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9519+0+0 (secure 0 0 0) 0x7fb3a80e63e0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.150+0000 7fb3b6a00640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 0x7fb39c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.151+0000 7fb3b6a00640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 0x7fb39c0b7990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8107500 tx=0x7fb3a40058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.152+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb3a80acba0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.386+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b802e5c0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.387+0000 7fb3afe00640 1 -- 172.21.15.67:0/1769770491 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11906 (secure 0 0 0) 0x7fb3a80ac4a0 con 0x7fb3b8102dc0 2024-09-10T23:16:20.388 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:16:20.388 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"btime":"2024-09-10T23:14:52:659129+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:14:52.238931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:16:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 33 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 msgr2=0x7fb39c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 0x7fb39c0b7990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8107500 tx=0x7fb3a40058d0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 msgr2=0x7fb3b8076f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800d950 tx=0x7fb3a800de10 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 shutdown_connections 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb39c0b54d0 0x7fb39c0b7990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3b8109150 0x7fb3b8075ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3b81087d0 0x7fb3b80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 --2- 172.21.15.67:0/1769770491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b8102dc0 0x7fb3b8076f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 >> 172.21.15.67:0/1769770491 conn(0x7fb3b80fe830 msgr2=0x7fb3b8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 shutdown_connections 2024-09-10T23:16:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:20.390+0000 7fb3be400640 1 -- 172.21.15.67:0/1769770491 wait complete. 2024-09-10T23:16:20.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1769770491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:20.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1769770491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:21.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1769770491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:21.560 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 54e8cfa9 --release --await' 2024-09-10T23:16:21.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 54e8cfa9 --release --await 2024-09-10T23:16:21.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:21 smithi145 ceph-mon[34397]: pgmap v933: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:16:21.941 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:21 smithi092 ceph-mon[34223]: pgmap v933: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:16:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:21 smithi067 ceph-mon[30939]: pgmap v933: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-10T23:16:23.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.180+0000 7f27aa800640 1 -- 172.21.15.67:0/609871267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a4109150 msgr2=0x7f27a410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.180+0000 7f27aa800640 1 --2- 172.21.15.67:0/609871267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a4109150 0x7f27a410f950 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f279c009ca0 tx=0x7f279c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 -- 172.21.15.67:0/609871267 shutdown_connections 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 --2- 172.21.15.67:0/609871267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a4109150 0x7f27a410f950 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 --2- 172.21.15.67:0/609871267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a41087d0 0x7f27a4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 --2- 172.21.15.67:0/609871267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4102dc0 0x7f27a41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 -- 172.21.15.67:0/609871267 >> 172.21.15.67:0/609871267 conn(0x7f27a40fe830 msgr2=0x7f27a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 -- 172.21.15.67:0/609871267 shutdown_connections 2024-09-10T23:16:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.181+0000 7f27aa800640 1 -- 172.21.15.67:0/609871267 wait complete. 2024-09-10T23:16:23.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.182+0000 7f27aa800640 1 Processor -- start 2024-09-10T23:16:23.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.182+0000 7f27aa800640 1 -- start start 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 0x7f27a419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4109150 0x7f27a41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4112d10 con 0x7f27a41087d0 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4112b90 con 0x7f27a4109150 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27aa800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4112e90 con 0x7f27a4102dc0 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f279be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f279be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49044/0 (socket says 172.21.15.67:49044) 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f279be00640 1 -- 172.21.15.67:0/3480112412 learned_addr learned my addr 172.21.15.67:0/3480112412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27a3400640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 0x7f27a419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.183+0000 7f27a3e00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4109150 0x7f27a41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 -- 172.21.15.67:0/3480112412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 msgr2=0x7f27a419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 0x7f27a419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 -- 172.21.15.67:0/3480112412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4109150 msgr2=0x7f27a41a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4109150 0x7f27a41a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 -- 172.21.15.67:0/3480112412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a41a1430 con 0x7f27a41087d0 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27a3400640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 0x7f27a419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f279be00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d770 tx=0x7f278c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27a0c00640 1 -- 172.21.15.67:0/3480112412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c017070 con 0x7f27a41087d0 2024-09-10T23:16:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27a0c00640 1 -- 172.21.15.67:0/3480112412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f278c0054b0 con 0x7f27a41087d0 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27a0c00640 1 -- 172.21.15.67:0/3480112412 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c004c60 con 0x7f27a41087d0 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a41a1720 con 0x7f27a41087d0 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.184+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a41a7de0 con 0x7f27a41087d0 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.187+0000 7f27ad1d3740 1 Processor -- start 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 -- start start 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b27ff0 0x55d998b283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b348a0 0x55d998b36c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d998a9f290 con 0x55d998b28910 2024-09-10T23:16:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d998a8e6c0 con 0x55d998b348a0 2024-09-10T23:16:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d998a924d0 con 0x55d998b27ff0 2024-09-10T23:16:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49050/0 (socket says 172.21.15.67:49050) 2024-09-10T23:16:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27a3400640 1 -- 172.21.15.67:0/3716477581 learned_addr learned my addr 172.21.15.67:0/3716477581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27a0c00640 1 -- 172.21.15.67:0/3480112412 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f278c004e00 con 0x7f27a41087d0 2024-09-10T23:16:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.188+0000 7f27a3e00640 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b27ff0 0x55d998b283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 -- 172.21.15.67:0/3716477581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b27ff0 msgr2=0x55d998b283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b27ff0 0x55d998b283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 -- 172.21.15.67:0/3716477581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b348a0 msgr2=0x55d998b36c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b348a0 0x55d998b36c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 -- 172.21.15.67:0/3716477581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d998a1b460 con 0x55d998b28910 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.189+0000 7f27a3400640 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f27940105b0 tx=0x7f2794010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caf198ce4b0e8d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f2798c00640 1 -- 172.21.15.67:0/3716477581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400e770 con 0x55d998b28910 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f2798c00640 1 -- 172.21.15.67:0/3716477581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f279400eda0 con 0x55d998b28910 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f2798c00640 1 -- 172.21.15.67:0/3716477581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279401ce10 con 0x55d998b28910 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f27a0c00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 0x7f27880b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f27ad1d3740 1 -- 172.21.15.67:0/3716477581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 msgr2=0x55d998b34360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.190+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f27940105b0 tx=0x7f2794010a70 comp rx=0 tx=0).stop 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 -- 172.21.15.67:0/3716477581 shutdown_connections 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b348a0 0x55d998b36c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27a0c00640 1 -- 172.21.15.67:0/3480112412 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9519+0+0 (secure 0 0 0) 0x7f278c0e5ca0 con 0x7f27a41087d0 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b34360 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/3716477581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b27ff0 0x55d998b283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 -- 172.21.15.67:0/3716477581 >> 172.21.15.67:0/3716477581 conn(0x55d998b25e20 msgr2=0x55d998b26fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27a3e00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 0x7f27880b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 -- 172.21.15.67:0/3716477581 shutdown_connections 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 -- 172.21.15.67:0/3716477581 wait complete. 2024-09-10T23:16:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27a3e00640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 0x7f27880b7990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f279c006010 tx=0x7f279c004a80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.191+0000 7f27ad1d3740 1 -- start start 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 Processor -- start 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998b32330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d9988efbb0 con 0x55d998b28910 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d998a296c0 con 0x55d998b27ff0 2024-09-10T23:16:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d998b37550 con 0x55d998b348a0 2024-09-10T23:16:23.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.192+0000 7f27ad1d3740 10 client.0 _ll_register_callbacks cb 0x55d998a44490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:23.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49066/0 (socket says 172.21.15.67:49066) 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f27a3400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 -- 172.21.15.67:0/4246176732 learned_addr learned my addr 172.21.15.67:0/4246176732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f27a3e00640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998b32330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 -- 172.21.15.67:0/4246176732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 msgr2=0x55d998b32330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998b32330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 -- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 msgr2=0x55d998b318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 -- 172.21.15.67:0/4246176732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9987ecb00 con 0x55d998b28910 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f27a3400640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f279be00640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f278c0e3070 tx=0x7f278c0e3510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b657f70bbc9f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.193+0000 7f2791600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c020070 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9519+0+0 (secure 0 0 0) 0x7f278c005120 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279400eda0 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f278c0ab900 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f27ad1d3740 10 client.16260 fetch_fsmap learned FSMap version 33 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d998863960 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.194+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f279400eda0 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f278c057ce0 con 0x55d998b28910 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 10 client.16260 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T23:16:23.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 10 client.16260 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 20 client.16260 populate_metadata read hostname 'smithi067' 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998b443d0 0x55d998b467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 4 client.16260 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d998aa11e0 con 0x55d998b443d0 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.195+0000 7f27a3400640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998b443d0 0x55d998b467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.196+0000 7f27a3400640 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998b443d0 0x55d998b467c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e274c5e5370cf66 server_cookie=d75e763517ceb7b9 in_seq=0 out_seq=0 2024-09-10T23:16:23.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.196+0000 7f2791600640 10 client.16260 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:16:23.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f2791600640 1 -- 172.21.15.67:0/4246176732 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d998aa11e0 con 0x55d998b443d0 2024-09-10T23:16:23.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f2791600640 10 client.16260 handle_command_reply: tid=0 2024-09-10T23:16:23.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 client.16260 shutdown 2024-09-10T23:16:23.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 msgr2=0x55d998b31df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f278c0e3070 tx=0x7f278c0e3510 comp rx=0 tx=0).stop 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 shutdown_connections 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998b443d0 0x55d998b467c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998b32330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b28910 0x55d998b31df0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/4246176732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 >> 172.21.15.67:0/4246176732 conn(0x55d998b25e20 msgr2=0x55d998b2b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f2791600640 0 client.16260 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.197+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 shutdown_connections 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.198+0000 7f27ad1d3740 1 -- 172.21.15.67:0/4246176732 wait complete. 2024-09-10T23:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.198+0000 7f27ad1d3740 20 client.16260 trim_cache size 0 max 16384 2024-09-10T23:16:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 Processor -- start 2024-09-10T23:16:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 -- start start 2024-09-10T23:16:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b20a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b28910 0x55d998b2eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b49e0 con 0x55d998b348a0 2024-09-10T23:16:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b2250 con 0x55d998b27ff0 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.216+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b00c0 con 0x55d998b28910 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49068/0 (socket says 172.21.15.67:49068) 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 -- 172.21.15.67:0/2986453974 learned_addr learned my addr 172.21.15.67:0/2986453974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f279be00640 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b28910 0x55d998b2eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3400640 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b20a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 -- 172.21.15.67:0/2986453974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b28910 msgr2=0x55d998b2eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b28910 0x55d998b2eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 -- 172.21.15.67:0/2986453974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 msgr2=0x55d998b20a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b20a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.217+0000 7f27a3e00640 1 -- 172.21.15.67:0/2986453974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d998b266a0 con 0x55d998b348a0 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.218+0000 7f27a3e00640 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f279c005fe0 tx=0x7f279c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a036d1ff5d50e23d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.218+0000 7f2793e00640 1 -- 172.21.15.67:0/2986453974 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c031e20 con 0x55d998b348a0 2024-09-10T23:16:23.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.218+0000 7f2793e00640 1 -- 172.21.15.67:0/2986453974 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f279c036050 con 0x55d998b348a0 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.218+0000 7f2793e00640 1 -- 172.21.15.67:0/2986453974 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c0043d0 con 0x55d998b348a0 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- 172.21.15.67:0/2986453974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 msgr2=0x55d998b2f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f279c005fe0 tx=0x7f279c044000 comp rx=0 tx=0).stop 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- 172.21.15.67:0/2986453974 shutdown_connections 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b348a0 0x55d998b2f000 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b28910 0x55d998b2eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/2986453974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b27ff0 0x55d998b20a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- 172.21.15.67:0/2986453974 >> 172.21.15.67:0/2986453974 conn(0x55d998b25e20 msgr2=0x55d998b30130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- 172.21.15.67:0/2986453974 shutdown_connections 2024-09-10T23:16:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- 172.21.15.67:0/2986453974 wait complete. 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 -- start start 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.219+0000 7f27ad1d3740 1 Processor -- start 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b28910 0x55d998b397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998baa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b0710 con 0x55d998b27ff0 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b1b80 con 0x55d998b28910 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b0de0 con 0x55d998b348a0 2024-09-10T23:16:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27ad1d3740 10 client.0 _ll_register_callbacks cb 0x7f278c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49084/0 (socket says 172.21.15.67:49084) 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27a3400640 1 -- 172.21.15.67:0/1527452410 learned_addr learned my addr 172.21.15.67:0/1527452410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f279be00640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b28910 0x55d998b397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.220+0000 7f27a3e00640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998baa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 -- 172.21.15.67:0/1527452410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 msgr2=0x55d998baa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998baa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 -- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b28910 msgr2=0x55d998b397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b28910 0x55d998b397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 -- 172.21.15.67:0/1527452410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d998b407b0 con 0x55d998b27ff0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27a3400640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f279400bbc0 tx=0x7f279401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f71ee56240c643 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f2791600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:16:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279401cd40 con 0x55d998b27ff0 2024-09-10T23:16:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9519+0+0 (secure 0 0 0) 0x7f2794004020 con 0x55d998b27ff0 2024-09-10T23:16:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.221+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279c036050 con 0x55d998b27ff0 2024-09-10T23:16:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2794037050 con 0x55d998b27ff0 2024-09-10T23:16:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 10 client.16266 fetch_fsmap learned FSMap version 33 2024-09-10T23:16:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d998b268a0 con 0x55d998b27ff0 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f279c036050 con 0x55d998b27ff0 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2794004660 con 0x55d998b27ff0 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 10 client.16266 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 10 client.16266 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 20 client.16266 populate_metadata read hostname 'smithi067' 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998bb1ef0 0x55d998bb42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 4 client.16266 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "54e8cfa9", "release": true, "await": true}] 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.222+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "54e8cfa9", "release": true, "await": true}) -- 0x55d998a9cc90 con 0x55d998bb1ef0 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.223+0000 7f279be00640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998bb1ef0 0x55d998bb42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.223+0000 7f279be00640 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998bb1ef0 0x55d998bb42e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c952361d45b855 server_cookie=a053552bfcae92b7 in_seq=0 out_seq=0 2024-09-10T23:16:23.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.223+0000 7f2791600640 10 client.16266 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f2791600640 1 -- 172.21.15.67:0/1527452410 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d998a9cc90 con 0x55d998bb1ef0 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f2791600640 10 client.16266 handle_command_reply: tid=0 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 client.16266 shutdown 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 msgr2=0x55d998b39240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f279400bbc0 tx=0x7f279401ca10 comp rx=0 tx=0).stop 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 shutdown_connections 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55d998bb1ef0 0x55d998bb42e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d998b348a0 0x55d998baa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d998b28910 0x55d998b397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 --2- 172.21.15.67:0/1527452410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d998b27ff0 0x55d998b39240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 >> 172.21.15.67:0/1527452410 conn(0x55d998b25e20 msgr2=0x55d998b3ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f2791600640 0 client.16266 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 shutdown_connections 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 1 -- 172.21.15.67:0/1527452410 wait complete. 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.260+0000 7f27ad1d3740 20 client.16266 trim_cache size 0 max 16384 2024-09-10T23:16:23.262 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 33, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 52, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "54e8cfa9": { 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 52, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:16:23.263 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 154.8, 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:16:23.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:16:23.265 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:16:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 msgr2=0x7f27880b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 0x7f27880b7990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f279c006010 tx=0x7f279c004a80 comp rx=0 tx=0).stop 2024-09-10T23:16:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 msgr2=0x7f27a419b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d770 tx=0x7f278c00dc30 comp rx=0 tx=0).stop 2024-09-10T23:16:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 shutdown_connections 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f27880b54d0 0x7f27880b7990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27a4109150 0x7f27a41a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27a41087d0 0x7f27a419b8c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 --2- 172.21.15.67:0/3480112412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27a4102dc0 0x7f27a419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.261+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 >> 172.21.15.67:0/3480112412 conn(0x7f27a40fe830 msgr2=0x7f27a4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.262+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 shutdown_connections 2024-09-10T23:16:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:16:23.262+0000 7f27aa800640 1 -- 172.21.15.67:0/3480112412 wait complete. 2024-09-10T23:16:23.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:23 smithi145 ceph-mon[34397]: pgmap v934: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T23:16:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:23 smithi092 ceph-mon[34223]: pgmap v934: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T23:16:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:23 smithi067 ceph-mon[30939]: pgmap v934: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-10T23:16:24.271 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '54e8cfa9', total seconds elapsed: 39.9 2024-09-10T23:16:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:25 smithi145 ceph-mon[34397]: pgmap v935: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 242 KiB/s wr, 6 op/s 2024-09-10T23:16:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:25 smithi092 ceph-mon[34223]: pgmap v935: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 242 KiB/s wr, 6 op/s 2024-09-10T23:16:26.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:25 smithi067 ceph-mon[30939]: pgmap v935: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 242 KiB/s wr, 6 op/s 2024-09-10T23:16:27.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:27 smithi145 ceph-mon[34397]: pgmap v936: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-10T23:16:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:27 smithi092 ceph-mon[34223]: pgmap v936: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-10T23:16:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:27 smithi067 ceph-mon[30939]: pgmap v936: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 6 op/s 2024-09-10T23:16:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:29 smithi145 ceph-mon[34397]: pgmap v937: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:16:30.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:29 smithi092 ceph-mon[34223]: pgmap v937: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:16:30.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:29 smithi067 ceph-mon[30939]: pgmap v937: 129 pgs: 129 active+clean; 8.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-09-10T23:16:31.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:31 smithi145 ceph-mon[34397]: pgmap v938: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-10T23:16:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:31 smithi092 ceph-mon[34223]: pgmap v938: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-10T23:16:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:31 smithi067 ceph-mon[30939]: pgmap v938: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.6 MiB/s wr, 9 op/s 2024-09-10T23:16:33.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:33 smithi145 ceph-mon[34397]: pgmap v939: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 8 op/s 2024-09-10T23:16:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:33 smithi092 ceph-mon[34223]: pgmap v939: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 8 op/s 2024-09-10T23:16:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:33 smithi067 ceph-mon[30939]: pgmap v939: 129 pgs: 129 active+clean; 9.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.9 MiB/s wr, 8 op/s 2024-09-10T23:16:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:35 smithi092 ceph-mon[34223]: pgmap v940: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:16:36.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:35 smithi067 ceph-mon[30939]: pgmap v940: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:16:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:35 smithi145 ceph-mon[34397]: pgmap v940: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-09-10T23:16:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:37 smithi092 ceph-mon[34223]: pgmap v941: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:37 smithi067 ceph-mon[30939]: pgmap v941: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:37 smithi145 ceph-mon[34397]: pgmap v941: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:39 smithi092 ceph-mon[34223]: pgmap v942: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:16:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:39 smithi067 ceph-mon[30939]: pgmap v942: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:16:40.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:39 smithi145 ceph-mon[34397]: pgmap v942: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:16:41.932 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:41 smithi145 ceph-mon[34397]: pgmap v943: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:16:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:41 smithi092 ceph-mon[34223]: pgmap v943: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:16:42.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:41 smithi067 ceph-mon[30939]: pgmap v943: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-09-10T23:16:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:43 smithi092 ceph-mon[34223]: pgmap v944: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:43 smithi067 ceph-mon[30939]: pgmap v944: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:43 smithi145 ceph-mon[34397]: pgmap v944: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:16:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:45 smithi092 ceph-mon[34223]: pgmap v945: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:16:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:45 smithi067 ceph-mon[30939]: pgmap v945: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:16:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:45 smithi145 ceph-mon[34397]: pgmap v945: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:16:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:47 smithi092 ceph-mon[34223]: pgmap v946: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:16:48.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:47 smithi067 ceph-mon[30939]: pgmap v946: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:16:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:47 smithi145 ceph-mon[34397]: pgmap v946: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:16:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:49 smithi092 ceph-mon[34223]: pgmap v947: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:16:50.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:49 smithi067 ceph-mon[30939]: pgmap v947: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:16:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:49 smithi145 ceph-mon[34397]: pgmap v947: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:16:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:51 smithi092 ceph-mon[34223]: pgmap v948: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:16:52.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:51 smithi067 ceph-mon[30939]: pgmap v948: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:16:52.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:51 smithi145 ceph-mon[34397]: pgmap v948: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:16:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:53 smithi092 ceph-mon[34223]: pgmap v949: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:16:54.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:53 smithi067 ceph-mon[30939]: pgmap v949: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:16:54.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:53 smithi145 ceph-mon[34397]: pgmap v949: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:16:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:55 smithi092 ceph-mon[34223]: pgmap v950: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:16:56.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:55 smithi067 ceph-mon[30939]: pgmap v950: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:16:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:55 smithi145 ceph-mon[34397]: pgmap v950: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-09-10T23:16:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:57 smithi092 ceph-mon[34223]: pgmap v951: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:16:58.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:57 smithi067 ceph-mon[30939]: pgmap v951: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:16:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:57 smithi145 ceph-mon[34397]: pgmap v951: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:17:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:16:59 smithi092 ceph-mon[34223]: pgmap v952: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:17:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:16:59 smithi067 ceph-mon[30939]: pgmap v952: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:17:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:16:59 smithi145 ceph-mon[34397]: pgmap v952: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:17:01.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:00 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:17:01.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:00 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:17:01.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:00 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:17:02.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:01 smithi092 ceph-mon[34223]: pgmap v953: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:01 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:17:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:01 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:17:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:01 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:17:02.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:01 smithi067 ceph-mon[30939]: pgmap v953: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:01 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:17:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:01 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:17:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:01 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:17:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:01 smithi145 ceph-mon[34397]: pgmap v953: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:01 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:17:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:01 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:17:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:01 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:17:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:03 smithi092 ceph-mon[34223]: pgmap v954: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:17:04.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:03 smithi067 ceph-mon[30939]: pgmap v954: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:17:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:03 smithi145 ceph-mon[34397]: pgmap v954: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 26 op/s 2024-09-10T23:17:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:05 smithi092 ceph-mon[34223]: pgmap v955: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:17:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:05 smithi067 ceph-mon[30939]: pgmap v955: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:17:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:05 smithi145 ceph-mon[34397]: pgmap v955: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-09-10T23:17:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:07 smithi092 ceph-mon[34223]: pgmap v956: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:17:08.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:07 smithi067 ceph-mon[30939]: pgmap v956: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:17:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:07 smithi145 ceph-mon[34397]: pgmap v956: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 32 op/s 2024-09-10T23:17:09.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:08 smithi092 ceph-mon[34223]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T23:17:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:08 smithi067 ceph-mon[30939]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T23:17:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:08 smithi145 ceph-mon[34397]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T23:17:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:09 smithi092 ceph-mon[34223]: pgmap v958: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:10.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:09 smithi067 ceph-mon[30939]: pgmap v958: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:09 smithi145 ceph-mon[34397]: pgmap v958: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:12.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:11 smithi092 ceph-mon[34223]: pgmap v959: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:12.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:11 smithi067 ceph-mon[30939]: pgmap v959: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:12.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:11 smithi145 ceph-mon[34397]: pgmap v959: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 34 op/s 2024-09-10T23:17:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:13 smithi092 ceph-mon[34223]: pgmap v960: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-10T23:17:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:13 smithi092 ceph-mon[34223]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T23:17:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:13 smithi067 ceph-mon[30939]: pgmap v960: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-10T23:17:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:13 smithi067 ceph-mon[30939]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T23:17:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:13 smithi145 ceph-mon[34397]: pgmap v960: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-10T23:17:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:13 smithi145 ceph-mon[34397]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T23:17:16.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:15 smithi092 ceph-mon[34223]: pgmap v962: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:17:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:15 smithi067 ceph-mon[30939]: pgmap v962: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:17:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:15 smithi145 ceph-mon[34397]: pgmap v962: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 10 op/s 2024-09-10T23:17:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:17 smithi092 ceph-mon[34223]: pgmap v963: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:17:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:17 smithi067 ceph-mon[30939]: pgmap v963: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:17:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:17 smithi145 ceph-mon[34397]: pgmap v963: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:17:19.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:18 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:19.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:18 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:19.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:18 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:19 smithi092 ceph-mon[34223]: pgmap v964: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:20.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:19 smithi067 ceph-mon[30939]: pgmap v964: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:20.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:19 smithi145 ceph-mon[34397]: pgmap v964: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:21 smithi092 ceph-mon[34223]: pgmap v965: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:17:22.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:21 smithi067 ceph-mon[30939]: pgmap v965: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:17:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:21 smithi145 ceph-mon[34397]: pgmap v965: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:17:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:23 smithi092 ceph-mon[34223]: pgmap v966: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:17:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:23 smithi067 ceph-mon[30939]: pgmap v966: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:17:24.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:23 smithi145 ceph-mon[34397]: pgmap v966: 129 pgs: 129 active+clean; 9.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:17:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:25 smithi092 ceph-mon[34223]: pgmap v967: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:17:26.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:25 smithi067 ceph-mon[30939]: pgmap v967: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:17:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:25 smithi145 ceph-mon[34397]: pgmap v967: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:17:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:27 smithi092 ceph-mon[34223]: pgmap v968: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:27 smithi067 ceph-mon[30939]: pgmap v968: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:28.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:27 smithi145 ceph-mon[34397]: pgmap v968: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:28.901 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:17:28.901 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-10T23:17:28.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:29.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:28 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:29.028 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:28 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:29.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:28 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:17:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:29 smithi092 ceph-mon[34223]: pgmap v969: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:17:30.119 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:29 smithi067 ceph-mon[30939]: pgmap v969: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:17:30.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:29 smithi145 ceph-mon[34397]: pgmap v969: 129 pgs: 129 active+clean; 9.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:17:30.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 -- 172.21.15.67:0/3387698529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 msgr2=0x7fd33c10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:30.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 --2- 172.21.15.67:0/3387698529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c10c9a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd328009900 tx=0x7fd32802a410 comp rx=0 tx=0).stop 2024-09-10T23:17:30.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 -- 172.21.15.67:0/3387698529 shutdown_connections 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 --2- 172.21.15.67:0/3387698529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 --2- 172.21.15.67:0/3387698529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c10c9a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 --2- 172.21.15.67:0/3387698529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 0x7fd33c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 -- 172.21.15.67:0/3387698529 >> 172.21.15.67:0/3387698529 conn(0x7fd33c0fd5b0 msgr2=0x7fd33c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 -- 172.21.15.67:0/3387698529 shutdown_connections 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.606+0000 7fd33be00640 1 -- 172.21.15.67:0/3387698529 wait complete. 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.607+0000 7fd33be00640 1 Processor -- start 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.607+0000 7fd33be00640 1 -- start start 2024-09-10T23:17:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 0x7fd33c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c112ad0 con 0x7fd33c1043c0 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c112950 con 0x7fd33c10cf70 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c112c50 con 0x7fd33c1039c0 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53478/0 (socket says 172.21.15.67:53478) 2024-09-10T23:17:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33a000640 1 -- 172.21.15.67:0/462670757 learned_addr learned my addr 172.21.15.67:0/462670757 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.608+0000 7fd33aa00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 0x7fd33c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 -- 172.21.15.67:0/462670757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 msgr2=0x7fd33c101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 0x7fd33c101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 -- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 msgr2=0x7fd33c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 -- 172.21.15.67:0/462670757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c1a7380 con 0x7fd33c10cf70 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33a000640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd33b400640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c1022c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd334009fd0 tx=0x7fd33400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.609+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33400ce20 con 0x7fd33c10cf70 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.610+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c1a7670 con 0x7fd33c10cf70 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.610+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd334005020 con 0x7fd33c10cf70 2024-09-10T23:17:30.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.610+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334010690 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.610+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c1a7bb0 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.611+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33c103da0 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.615+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd334004a40 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.616+0000 7fd333400640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 0x7fd3200b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.617+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fd3340e6a40 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.617+0000 7fd33aa00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 0x7fd3200b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.617+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd3340e6e20 con 0x7fd33c10cf70 2024-09-10T23:17:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.617+0000 7fd33aa00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 0x7fd3200b7a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00bbe0 tx=0x7fd32c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:30.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.921+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33c02e5c0 con 0x7fd33c10cf70 2024-09-10T23:17:30.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.923+0000 7fd333400640 1 -- 172.21.15.67:0/462670757 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fd3340ad180 con 0x7fd33c10cf70 2024-09-10T23:17:30.926 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:17:30.926 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:17:30.926 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:17:30.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.926+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 msgr2=0x7fd3200b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:30.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.926+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 0x7fd3200b7a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00bbe0 tx=0x7fd32c00b390 comp rx=0 tx=0).stop 2024-09-10T23:17:30.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.926+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 msgr2=0x7fd33c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:30.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.926+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c1022c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd334009fd0 tx=0x7fd33400ef10 comp rx=0 tx=0).stop 2024-09-10T23:17:30.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 shutdown_connections 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd3200b5580 0x7fd3200b7a40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd33c10cf70 0x7fd33c1022c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd33c1043c0 0x7fd33c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 --2- 172.21.15.67:0/462670757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd33c1039c0 0x7fd33c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 >> 172.21.15.67:0/462670757 conn(0x7fd33c0fd5b0 msgr2=0x7fd33c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 shutdown_connections 2024-09-10T23:17:30.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:30.927+0000 7fd33be00640 1 -- 172.21.15.67:0/462670757 wait complete. 2024-09-10T23:17:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:31 smithi092 ceph-mon[34223]: pgmap v970: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:31 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/462670757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:31 smithi145 ceph-mon[34397]: pgmap v970: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:31 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/462670757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:32.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:31 smithi067 ceph-mon[30939]: pgmap v970: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:17:32.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:31 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/462670757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:33.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-10T23:17:33.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-10T23:17:33.973 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:33 smithi067 ceph-mon[30939]: pgmap v971: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:33 smithi092 ceph-mon[34223]: pgmap v971: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:33 smithi145 ceph-mon[34397]: pgmap v971: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:34.369 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:35 smithi092 ceph-mon[34223]: pgmap v972: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:35 smithi067 ceph-mon[30939]: pgmap v972: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:35 smithi145 ceph-mon[34397]: pgmap v972: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1208913276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 msgr2=0x7ff9a4111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1208913276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4111610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009ca0 tx=0x7ff99c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1208913276 shutdown_connections 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1208913276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4111610 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1208913276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1208913276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1208913276 >> 172.21.15.67:0/1208913276 conn(0x7ff9a40fe660 msgr2=0x7ff9a4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1208913276 shutdown_connections 2024-09-10T23:17:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.623+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1208913276 wait complete. 2024-09-10T23:17:36.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.624+0000 7ff9a3e00640 1 Processor -- start 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 -- start start 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a4102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a4102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4114d00 con 0x7ff9a410f220 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4114b80 con 0x7ff9a40754c0 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4114e80 con 0x7ff9a4075de0 2024-09-10T23:17:36.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.625+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a4102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53490/0 (socket says 172.21.15.67:53490) 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a4102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:56260/0 (socket says 172.21.15.67:56260) 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a3400640 1 -- 172.21.15.67:0/1946956963 learned_addr learned my addr 172.21.15.67:0/1946956963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a4102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 -- 172.21.15.67:0/1946956963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 msgr2=0x7ff9a4102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a4102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 -- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 msgr2=0x7ff9a4103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a2a00640 1 -- 172.21.15.67:0/1946956963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a41ab740 con 0x7ff9a40754c0 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.626+0000 7ff9a3400640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a4102990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d6d0 tx=0x7ff99400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff99b400640 1 -- 172.21.15.67:0/1946956963 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994017070 con 0x7ff9a40754c0 2024-09-10T23:17:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a41aba30 con 0x7ff9a40754c0 2024-09-10T23:17:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff99b400640 1 -- 172.21.15.67:0/1946956963 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff994009820 con 0x7ff9a40754c0 2024-09-10T23:17:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff99b400640 1 -- 172.21.15.67:0/1946956963 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994004bc0 con 0x7ff9a40754c0 2024-09-10T23:17:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.627+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a41abfc0 con 0x7ff9a40754c0 2024-09-10T23:17:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 Processor -- start 2024-09-10T23:17:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 -- start start 2024-09-10T23:17:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 0x562b56645660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56645ba0 0x562b56647f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562b565aeca0 con 0x562b566392f0 2024-09-10T23:17:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b5659e0d0 con 0x562b56645ba0 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.632+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562b565a1ee0 con 0x562b56639c10 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff99b400640 1 -- 172.21.15.67:0/1946956963 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff994005220 con 0x7ff9a40754c0 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53500/0 (socket says 172.21.15.67:53500) 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 -- 172.21.15.67:0/3186478504 learned_addr learned my addr 172.21.15.67:0/3186478504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a2000640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56645ba0 0x562b56647f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 -- 172.21.15.67:0/3186478504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 msgr2=0x562b56645660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 0x562b56645660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 0x562b56645660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 -- 172.21.15.67:0/3186478504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56645ba0 msgr2=0x562b56647f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56645ba0 0x562b56647f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a3400640 1 -- 172.21.15.67:0/3186478504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b56550400 con 0x562b566392f0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.633+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 0x562b56645660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.634+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02aeb0 tx=0x7ff99c0318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b8226c38da8e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.634+0000 7ff98fe00640 1 -- 172.21.15.67:0/3186478504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c031cc0 con 0x562b566392f0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.634+0000 7ff98fe00640 1 -- 172.21.15.67:0/3186478504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff99c036030 con 0x562b566392f0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.634+0000 7ff98fe00640 1 -- 172.21.15.67:0/3186478504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c004750 con 0x562b566392f0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff99b400640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 0x7ff9880b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff99b400640 1 -- 172.21.15.67:0/1946956963 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7ff9940e5e20 con 0x7ff9a40754c0 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3186478504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 msgr2=0x562b566396d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02aeb0 tx=0x7ff99c0318b0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 0x7ff9880b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3186478504 shutdown_connections 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56645ba0 0x562b56647f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56639c10 0x562b56645660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3186478504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b566392f0 0x562b566396d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3186478504 >> 172.21.15.67:0/3186478504 conn(0x562b56637120 msgr2=0x562b566382c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.635+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3186478504 shutdown_connections 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.636+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3186478504 wait complete. 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.636+0000 7ff9a2a00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 0x7ff9880b7a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x562b56638000 tx=0x7ff994000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.636+0000 7ff9ac1e8740 1 -- start start 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.636+0000 7ff9ac1e8740 1 Processor -- start 2024-09-10T23:17:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 0x562b56645480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56645ba0 0x562b56642f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562b563ff7a0 con 0x562b56639c10 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b56543a10 con 0x562b566392f0 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562b56648640 con 0x562b56645ba0 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9ac1e8740 10 client.0 _ll_register_callbacks cb 0x562b56553be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9a2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9a2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53504/0 (socket says 172.21.15.67:53504) 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9a2000640 1 -- 172.21.15.67:0/2943789775 learned_addr learned my addr 172.21.15.67:0/2943789775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.637+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 0x562b56645480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a3400640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56645ba0 0x562b56642f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a2000640 1 -- 172.21.15.67:0/2943789775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56645ba0 msgr2=0x562b56642f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a2000640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56645ba0 0x562b56642f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a2000640 1 -- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 msgr2=0x562b56645480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a2000640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 0x562b56645480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a2000640 1 -- 172.21.15.67:0/2943789775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b5630bb90 con 0x562b56639c10 2024-09-10T23:17:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.638+0000 7ff9a3400640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 0x562b56645480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff9a2000640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff990012700 tx=0x7ff990012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7257357c2d95a75f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff98cc00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:17:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99000f8a0 con 0x562b56639c10 2024-09-10T23:17:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99c036030 con 0x562b56639c10 2024-09-10T23:17:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7ff99000fa40 con 0x562b56639c10 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff99000fc40 con 0x562b56639c10 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff9ac1e8740 10 client.16284 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.639+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b56653fb0 con 0x562b56639c10 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff99c036030 con 0x562b56639c10 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff99001c420 con 0x562b56639c10 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 10 client.16284 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 10 client.16284 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 20 client.16284 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566553c0 0x562b56657820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 4 client.16284 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.640+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b565b0bf0 con 0x562b566553c0 2024-09-10T23:17:36.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.641+0000 7ff9a3400640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566553c0 0x562b56657820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.641+0000 7ff9a3400640 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566553c0 0x562b56657820 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac83ad6f400da86 server_cookie=51f82ec931afbbc7 in_seq=0 out_seq=0 2024-09-10T23:17:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.641+0000 7ff98cc00640 10 client.16284 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff98cc00640 1 -- 172.21.15.67:0/2943789775 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b565b0bf0 con 0x562b566553c0 2024-09-10T23:17:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff98cc00640 10 client.16284 handle_command_reply: tid=0 2024-09-10T23:17:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff9ac1e8740 1 client.16284 shutdown 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 msgr2=0x562b566428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff990012700 tx=0x7ff990012bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 shutdown_connections 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.642+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566553c0 0x562b56657820 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b56645ba0 0x562b56642f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56639c10 0x562b566428d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/2943789775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b566392f0 0x562b56645480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 >> 172.21.15.67:0/2943789775 conn(0x562b56637120 msgr2=0x562b5663cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff98cc00640 0 client.16284 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 shutdown_connections 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/2943789775 wait complete. 2024-09-10T23:17:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.643+0000 7ff9ac1e8740 20 client.16284 trim_cache size 0 max 16384 2024-09-10T23:17:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 Processor -- start 2024-09-10T23:17:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 -- start start 2024-09-10T23:17:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b56636af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b56646f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b8ae0 con 0x562b56645ba0 2024-09-10T23:17:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b63b0 con 0x562b56639c10 2024-09-10T23:17:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.664+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b4220 con 0x562b566392f0 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53508/0 (socket says 172.21.15.67:53508) 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b56646f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a3400640 1 -- 172.21.15.67:0/170165623 learned_addr learned my addr 172.21.15.67:0/170165623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a3400640 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b56636af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 -- 172.21.15.67:0/170165623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 msgr2=0x562b56636af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b56636af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 -- 172.21.15.67:0/170165623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 msgr2=0x562b566474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 -- 172.21.15.67:0/170165623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b56638cd0 con 0x562b56639c10 2024-09-10T23:17:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a3400640 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.665+0000 7ff9a2000640 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b56646f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9900059f0 tx=0x7ff990005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3036e1de083f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.666+0000 7ff98f400640 1 -- 172.21.15.67:0/170165623 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990009e20 con 0x562b56639c10 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.666+0000 7ff98f400640 1 -- 172.21.15.67:0/170165623 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff99000f8a0 con 0x562b56639c10 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.666+0000 7ff98f400640 1 -- 172.21.15.67:0/170165623 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99001cd10 con 0x562b56639c10 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/170165623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 msgr2=0x562b56646f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b56646f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9900059f0 tx=0x7ff990005d30 comp rx=0 tx=0).stop 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/170165623 shutdown_connections 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b566474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b56646f90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/170165623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b56636af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/170165623 >> 172.21.15.67:0/170165623 conn(0x562b56637120 msgr2=0x562b56641150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/170165623 shutdown_connections 2024-09-10T23:17:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/170165623 wait complete. 2024-09-10T23:17:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.667+0000 7ff9ac1e8740 1 -- start start 2024-09-10T23:17:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 Processor -- start 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b5664e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b566488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b4870 con 0x562b56645ba0 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b5ce0 con 0x562b56639c10 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b4f40 con 0x562b566392f0 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.668+0000 7ff9ac1e8740 10 client.0 _ll_register_callbacks cb 0x7ff990009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53522/0 (socket says 172.21.15.67:53522) 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 -- 172.21.15.67:0/3372761299 learned_addr learned my addr 172.21.15.67:0/3372761299 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a2000640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b566488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b5664e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 -- 172.21.15.67:0/3372761299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 msgr2=0x562b5664e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b5664e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 -- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 msgr2=0x562b566488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b566488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 -- 172.21.15.67:0/3372761299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b566c22f0 con 0x562b56645ba0 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a2000640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b566488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff9a3400640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02ae80 tx=0x7ff99c056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d057dfb830e53962 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.669+0000 7ff98cc00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:17:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c032e90 con 0x562b56645ba0 2024-09-10T23:17:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7ff99c003970 con 0x562b56645ba0 2024-09-10T23:17:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99000f8a0 con 0x562b56645ba0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff99c066050 con 0x562b56645ba0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff9ac1e8740 10 client.16290 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.670+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b566c35a0 con 0x562b56645ba0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff99000f8a0 con 0x562b56645ba0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff99c043070 con 0x562b56645ba0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 10 client.16290 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 10 client.16290 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 20 client.16290 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566c3ab0 0x562b566c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 4 client.16290 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.671+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x562b565ac6a0 con 0x562b566c3ab0 2024-09-10T23:17:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.672+0000 7ff9a2000640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566c3ab0 0x562b566c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.672+0000 7ff9a2000640 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566c3ab0 0x562b566c5ea0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610b7a9fed12bc64 server_cookie=56018f7528ddf749 in_seq=0 out_seq=0 2024-09-10T23:17:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.672+0000 7ff98cc00640 10 client.16290 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.811+0000 7ff98cc00640 1 -- 172.21.15.67:0/3372761299 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562b565ac6a0 con 0x562b566c3ab0 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.811+0000 7ff98cc00640 10 client.16290 handle_command_reply: tid=0 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 client.16290 shutdown 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 msgr2=0x562b56649070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02ae80 tx=0x7ff99c056040 comp rx=0 tx=0).stop 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 shutdown_connections 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562b566c3ab0 0x562b566c5ea0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562b56645ba0 0x562b56649070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b56639c10 0x562b566488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 --2- 172.21.15.67:0/3372761299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562b566392f0 0x562b5664e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 >> 172.21.15.67:0/3372761299 conn(0x562b56637120 msgr2=0x562b56643f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff98cc00640 0 client.16290 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 shutdown_connections 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 1 -- 172.21.15.67:0/3372761299 wait complete. 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.812+0000 7ff9ac1e8740 20 client.16290 trim_cache size 0 max 16384 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:17:36.815 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 54, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "ee1bb40a": { 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 54, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.0, 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:17:36.816 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:36.817 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 msgr2=0x7ff9880b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 0x7ff9880b7a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x562b56638000 tx=0x7ff994000bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 msgr2=0x7ff9a4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a4102990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d6d0 tx=0x7ff99400db90 comp rx=0 tx=0).stop 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 shutdown_connections 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff9880b5580 0x7ff9880b7a40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a410f220 0x7ff9a4103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a4075de0 0x7ff9a4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 --2- 172.21.15.67:0/1946956963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40754c0 0x7ff9a4102990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 >> 172.21.15.67:0/1946956963 conn(0x7ff9a40fe660 msgr2=0x7ff9a41113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 shutdown_connections 2024-09-10T23:17:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:36.813+0000 7ff9a3e00640 1 -- 172.21.15.67:0/1946956963 wait complete. 2024-09-10T23:17:37.657 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee1bb40a', quiesce took 8.8 seconds. Will release after: 12.2 2024-09-10T23:17:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:37 smithi067 ceph-mon[30939]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:17:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:37 smithi145 ceph-mon[34397]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:17:38.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:37 smithi092 ceph-mon[34223]: pgmap v973: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:17:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:39 smithi067 ceph-mon[30939]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:39 smithi145 ceph-mon[34397]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:39 smithi092 ceph-mon[34223]: pgmap v974: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:17:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:41 smithi067 ceph-mon[30939]: pgmap v975: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:41 smithi145 ceph-mon[34397]: pgmap v975: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:41 smithi092 ceph-mon[34223]: pgmap v975: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:17:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:43 smithi067 ceph-mon[30939]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:17:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:43 smithi145 ceph-mon[34397]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:17:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:43 smithi092 ceph-mon[34223]: pgmap v976: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.0 MiB/s wr, 11 op/s 2024-09-10T23:17:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:45 smithi067 ceph-mon[30939]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-09-10T23:17:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:45 smithi145 ceph-mon[34397]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-09-10T23:17:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:45 smithi092 ceph-mon[34223]: pgmap v977: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.3 MiB/s wr, 11 op/s 2024-09-10T23:17:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:47 smithi067 ceph-mon[30939]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 11 op/s 2024-09-10T23:17:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:47 smithi145 ceph-mon[34397]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 11 op/s 2024-09-10T23:17:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:47 smithi092 ceph-mon[34223]: pgmap v978: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 11 op/s 2024-09-10T23:17:49.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:49 smithi067 ceph-mon[30939]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 8 op/s 2024-09-10T23:17:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:49 smithi145 ceph-mon[34397]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 8 op/s 2024-09-10T23:17:50.255 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:49 smithi092 ceph-mon[34223]: pgmap v979: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 8 op/s 2024-09-10T23:17:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 -- 172.21.15.67:0/876264123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 msgr2=0x7fa2d8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 --2- 172.21.15.67:0/876264123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d8102ff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009900 tx=0x7fa2cc02a410 comp rx=0 tx=0).stop 2024-09-10T23:17:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 -- 172.21.15.67:0/876264123 shutdown_connections 2024-09-10T23:17:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 --2- 172.21.15.67:0/876264123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 0x7fa2d810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 --2- 172.21.15.67:0/876264123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 0x7fa2d8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 --2- 172.21.15.67:0/876264123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d8102ff0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 -- 172.21.15.67:0/876264123 >> 172.21.15.67:0/876264123 conn(0x7fa2d80fe660 msgr2=0x7fa2d8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 -- 172.21.15.67:0/876264123 shutdown_connections 2024-09-10T23:17:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:50.999+0000 7fa2dea00640 1 -- 172.21.15.67:0/876264123 wait complete. 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.000+0000 7fa2dea00640 1 Processor -- start 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.000+0000 7fa2dea00640 1 -- start start 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 0x7fa2d80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 0x7fa2d8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112b10 con 0x7fa2d8102c10 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112990 con 0x7fa2d8108fa0 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112c90 con 0x7fa2d8108620 2024-09-10T23:17:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33064/0 (socket says 172.21.15.67:33064) 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 -- 172.21.15.67:0/3025565478 learned_addr learned my addr 172.21.15.67:0/3025565478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2de000640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 0x7fa2d8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dcc00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 0x7fa2d80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 -- 172.21.15.67:0/3025565478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 msgr2=0x7fa2d80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 0x7fa2d80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 -- 172.21.15.67:0/3025565478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 msgr2=0x7fa2d8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 0x7fa2d8075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.001+0000 7fa2dd600640 1 -- 172.21.15.67:0/3025565478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d81a7380 con 0x7fa2d8102c10 2024-09-10T23:17:51.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2dd600640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009230 tx=0x7fa2cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc03f030 con 0x7fa2d8102c10 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2cc003a80 con 0x7fa2d8102c10 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d81a7610 con 0x7fa2d8102c10 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc035070 con 0x7fa2d8102c10 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.002+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a7b40 con 0x7fa2d8102c10 2024-09-10T23:17:51.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.003+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d8102ff0 con 0x7fa2d8102c10 2024-09-10T23:17:51.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.006+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa2cc03bc30 con 0x7fa2d8102c10 2024-09-10T23:17:51.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.008+0000 7fa2d6000640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 0x7fa2c00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:51.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.009+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fa2cc107160 con 0x7fa2d8102c10 2024-09-10T23:17:51.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.009+0000 7fa2dcc00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 0x7fa2c00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:51.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.009+0000 7fa2dcc00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 0x7fa2c00b7a40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8076f90 tx=0x7fa2c400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:51.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.010+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2cc0cd250 con 0x7fa2d8102c10 2024-09-10T23:17:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.247+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d80767a0 con 0x7fa2d8102c10 2024-09-10T23:17:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.248+0000 7fa2d6000640 1 -- 172.21.15.67:0/3025565478 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fa2d80767a0 con 0x7fa2d8102c10 2024-09-10T23:17:51.248 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:17:51.249 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:17:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 msgr2=0x7fa2c00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 0x7fa2c00b7a40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8076f90 tx=0x7fa2c400b390 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 msgr2=0x7fa2d80754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009230 tx=0x7fa2cc004270 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 shutdown_connections 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa2c00b5580 0x7fa2c00b7a40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d8108fa0 0x7fa2d8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8108620 0x7fa2d80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 --2- 172.21.15.67:0/3025565478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2d8102c10 0x7fa2d80754a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.250+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 >> 172.21.15.67:0/3025565478 conn(0x7fa2d80fe660 msgr2=0x7fa2d8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.251+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 shutdown_connections 2024-09-10T23:17:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:51.251+0000 7fa2dea00640 1 -- 172.21.15.67:0/3025565478 wait complete. 2024-09-10T23:17:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:51 smithi145 ceph-mon[34397]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.5 MiB/s wr, 9 op/s 2024-09-10T23:17:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:51 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3025565478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:51 smithi092 ceph-mon[34223]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.5 MiB/s wr, 9 op/s 2024-09-10T23:17:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:51 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3025565478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:51 smithi067 ceph-mon[30939]: pgmap v980: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.5 MiB/s wr, 9 op/s 2024-09-10T23:17:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:51 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3025565478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:52.368 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id ee1bb40a --release --await' 2024-09-10T23:17:52.368 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id ee1bb40a --release --await 2024-09-10T23:17:52.760 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 -- 172.21.15.67:0/2257232408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075f40 msgr2=0x7fc2d410edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 --2- 172.21.15.67:0/2257232408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075f40 0x7fc2d410edf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009900 tx=0x7fc2bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 -- 172.21.15.67:0/2257232408 shutdown_connections 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 --2- 172.21.15.67:0/2257232408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 0x7fc2d41117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 --2- 172.21.15.67:0/2257232408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075f40 0x7fc2d410edf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 --2- 172.21.15.67:0/2257232408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075540 0x7fc2d4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 -- 172.21.15.67:0/2257232408 >> 172.21.15.67:0/2257232408 conn(0x7fc2d40fe810 msgr2=0x7fc2d4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 -- 172.21.15.67:0/2257232408 shutdown_connections 2024-09-10T23:17:53.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.501+0000 7fc2db200640 1 -- 172.21.15.67:0/2257232408 wait complete. 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.502+0000 7fc2db200640 1 Processor -- start 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 -- start start 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075f40 0x7fc2d419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 0x7fc2d41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4114e80 con 0x7fc2d4075540 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4114d00 con 0x7fc2d4075f40 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2db200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4115000 con 0x7fc2d410f3c0 2024-09-10T23:17:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33086/0 (socket says 172.21.15.67:33086) 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2d3e00640 1 -- 172.21.15.67:0/1345770983 learned_addr learned my addr 172.21.15.67:0/1345770983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2d9200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 0x7fc2d41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.503+0000 7fc2cb400640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075f40 0x7fc2d419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 -- 172.21.15.67:0/1345770983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 msgr2=0x7fc2d41a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 0x7fc2d41a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 -- 172.21.15.67:0/1345770983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075f40 msgr2=0x7fc2d419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075f40 0x7fc2d419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 -- 172.21.15.67:0/1345770983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d41a14c0 con 0x7fc2d4075540 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d950 tx=0x7fc2c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d1600640 1 -- 172.21.15.67:0/1345770983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017030 con 0x7fc2d4075540 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d1600640 1 -- 172.21.15.67:0/1345770983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c4004020 con 0x7fc2d4075540 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2d1600640 1 -- 172.21.15.67:0/1345770983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40055d0 con 0x7fc2d4075540 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.504+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d41a17b0 con 0x7fc2d4075540 2024-09-10T23:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.505+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41a7e70 con 0x7fc2d4075540 2024-09-10T23:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 Processor -- start 2024-09-10T23:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 -- start start 2024-09-10T23:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2d1600640 1 -- 172.21.15.67:0/1345770983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc2c4009a30 con 0x7fc2d4075540 2024-09-10T23:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 0x557d1e5cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557d1e537290 con 0x557d1e5bfff0 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d1e5266c0 con 0x557d1e5cc8a0 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557d1e52a4d0 con 0x557d1e5c0910 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.508+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33100/0 (socket says 172.21.15.67:33100) 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 -- 172.21.15.67:0/1454373286 learned_addr learned my addr 172.21.15.67:0/1454373286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2cb400640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 0x557d1e5cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 -- 172.21.15.67:0/1454373286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 msgr2=0x557d1e5cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 -- 172.21.15.67:0/1454373286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 msgr2=0x557d1e5cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 0x557d1e5cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 -- 172.21.15.67:0/1454373286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1e4b3460 con 0x557d1e5bfff0 2024-09-10T23:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2cb400640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 0x557d1e5cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.509+0000 7fc2d9200640 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009e60 tx=0x7fc2cc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772cfd49789f963c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2c9600640 1 -- 172.21.15.67:0/1454373286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016070 con 0x557d1e5bfff0 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2c9600640 1 -- 172.21.15.67:0/1454373286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2cc004d20 con 0x557d1e5bfff0 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2c9600640 1 -- 172.21.15.67:0/1454373286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc012650 con 0x557d1e5bfff0 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2d1600640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 0x7fc2b80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2d1600640 1 -- 172.21.15.67:0/1345770983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fc2c40e5ed0 con 0x7fc2d4075540 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.510+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 0x7fc2b80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/1454373286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 msgr2=0x557d1e5c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009e60 tx=0x7fc2cc010740 comp rx=0 tx=0).stop 2024-09-10T23:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/1454373286 shutdown_connections 2024-09-10T23:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5cc8a0 0x557d1e5cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/1454373286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bfff0 0x557d1e5c03d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/1454373286 >> 172.21.15.67:0/1454373286 conn(0x557d1e5bde20 msgr2=0x557d1e5befc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/1454373286 shutdown_connections 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/1454373286 wait complete. 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2d3e00640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 0x7fc2b80b7990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x557d1e5c3170 tx=0x7fc2c40e9980 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.511+0000 7fc2ddcc4740 1 -- start start 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 Processor -- start 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5c0910 0x557d1e5c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557d1e387bb0 con 0x557d1e5cc8a0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d1e4c16c0 con 0x557d1e5c0910 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557d1e5cf550 con 0x557d1e5bfff0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2ddcc4740 10 client.0 _ll_register_callbacks cb 0x557d1e4dc490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.512+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33108/0 (socket says 172.21.15.67:33108) 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 -- 172.21.15.67:0/55403741 learned_addr learned my addr 172.21.15.67:0/55403741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2cb400640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5c0910 0x557d1e5c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 -- 172.21.15.67:0/55403741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 msgr2=0x557d1e5c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 -- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5c0910 msgr2=0x557d1e5c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5c0910 0x557d1e5c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 -- 172.21.15.67:0/55403741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1e284b00 con 0x557d1e5cc8a0 2024-09-10T23:17:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2d9200640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0126f0 tx=0x7fc2cc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5fe47bbba4400c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.513+0000 7fc2c2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:17:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016040 con 0x557d1e5cc8a0 2024-09-10T23:17:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fc2cc012ac0 con 0x557d1e5cc8a0 2024-09-10T23:17:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2cc004d20 con 0x557d1e5cc8a0 2024-09-10T23:17:53.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2cc004ed0 con 0x557d1e5cc8a0 2024-09-10T23:17:53.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2ddcc4740 10 client.16308 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557d1e2fb960 con 0x557d1e5cc8a0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2cc004d20 con 0x557d1e5cc8a0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.514+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc2cc00f040 con 0x557d1e5cc8a0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 10 client.16308 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 10 client.16308 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 20 client.16308 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5dc3f0 0x557d1e5de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 4 client.16308 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1e5391e0 con 0x557d1e5dc3f0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.515+0000 7fc2cb400640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5dc3f0 0x557d1e5de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.516+0000 7fc2cb400640 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5dc3f0 0x557d1e5de7e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d569e19ddf127d12 server_cookie=131ffc43e5560ced in_seq=0 out_seq=0 2024-09-10T23:17:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.516+0000 7fc2c2000640 10 client.16308 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2c2000640 1 -- 172.21.15.67:0/55403741 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d1e5391e0 con 0x557d1e5dc3f0 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2c2000640 10 client.16308 handle_command_reply: tid=0 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 client.16308 shutdown 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 msgr2=0x557d1e5ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0126f0 tx=0x7fc2cc005a20 comp rx=0 tx=0).stop 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 shutdown_connections 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5dc3f0 0x557d1e5de7e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5cc8a0 0x557d1e5ca330 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5c0910 0x557d1e5c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/55403741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 >> 172.21.15.67:0/55403741 conn(0x557d1e5bde20 msgr2=0x557d1e5c39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2c2000640 0 client.16308 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 shutdown_connections 2024-09-10T23:17:53.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/55403741 wait complete. 2024-09-10T23:17:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.517+0000 7fc2ddcc4740 20 client.16308 trim_cache size 0 max 16384 2024-09-10T23:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.535+0000 7fc2ddcc4740 1 Processor -- start 2024-09-10T23:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 -- start start 2024-09-10T23:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bde20 0x557d1e5de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b4980 con 0x557d1e5c0910 2024-09-10T23:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b2250 con 0x557d1e5bde20 2024-09-10T23:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b00c0 con 0x557d1e5bfff0 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33110/0 (socket says 172.21.15.67:33110) 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2d9200640 1 -- 172.21.15.67:0/76199690 learned_addr learned my addr 172.21.15.67:0/76199690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2cb400640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.536+0000 7fc2d9200640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bde20 0x557d1e5de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 -- 172.21.15.67:0/76199690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bde20 msgr2=0x557d1e5de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bde20 0x557d1e5de400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 -- 172.21.15.67:0/76199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 msgr2=0x557d1e5e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 -- 172.21.15.67:0/76199690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1e5c51e0 con 0x557d1e5bfff0 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2d9200640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2cb400640 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5bcca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc02a910 tx=0x7fc2bc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ac9e6810f6ae7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2c8c00640 1 -- 172.21.15.67:0/76199690 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc048020 con 0x557d1e5bfff0 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2c8c00640 1 -- 172.21.15.67:0/76199690 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2bc0046c0 con 0x557d1e5bfff0 2024-09-10T23:17:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.537+0000 7fc2c8c00640 1 -- 172.21.15.67:0/76199690 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035040 con 0x557d1e5bfff0 2024-09-10T23:17:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/76199690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 msgr2=0x557d1e5bcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5bcca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc02a910 tx=0x7fc2bc000bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/76199690 shutdown_connections 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5c0910 0x557d1e5e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5bfff0 0x557d1e5bcca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/76199690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bde20 0x557d1e5de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/76199690 >> 172.21.15.67:0/76199690 conn(0x557d1e5deae0 msgr2=0x557d1e5bc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/76199690 shutdown_connections 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.538+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/76199690 wait complete. 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 -- start start 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 Processor -- start 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 0x557d1e5d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b0710 con 0x557d1e5bde20 2024-09-10T23:17:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b1b80 con 0x557d1e5bfff0 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.539+0000 7fc2ddcc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41b0de0 con 0x557d1e5c0910 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2ddcc4740 10 client.0 _ll_register_callbacks cb 0x7fc2c40e9850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33114/0 (socket says 172.21.15.67:33114) 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 -- 172.21.15.67:0/3726178642 learned_addr learned my addr 172.21.15.67:0/3726178642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2cb400640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 0x557d1e5d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 -- 172.21.15.67:0/3726178642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 msgr2=0x557d1e5cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 -- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 msgr2=0x557d1e5d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 0x557d1e5d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2d9200640 1 -- 172.21.15.67:0/3726178642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1e5cfce0 con 0x557d1e5bde20 2024-09-10T23:17:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.540+0000 7fc2cb400640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 0x557d1e5d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2d9200640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009e60 tx=0x7fc2cc009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e2ec3db27fd247 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2c2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016040 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc0046c0 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fc2cc004cd0 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2cc010c40 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2ddcc4740 10 client.16314 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557d1e6499b0 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.541+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2bc0046c0 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc2cc0055b0 con 0x557d1e5bde20 2024-09-10T23:17:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 10 client.16314 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 10 client.16314 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 20 client.16314 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5cc8a0 0x557d1e64c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 4 client.16314 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee1bb40a", "release": true, "await": true}] 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee1bb40a", "release": true, "await": true}) -- 0x557d1e534c90 con 0x557d1e5cc8a0 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.542+0000 7fc2cb400640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5cc8a0 0x557d1e64c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.543+0000 7fc2cb400640 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5cc8a0 0x557d1e64c080 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb0f5bf62ebde37 server_cookie=c705cacb7e761df6 in_seq=0 out_seq=0 2024-09-10T23:17:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.543+0000 7fc2c2000640 10 client.16314 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:53.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2c2000640 1 -- 172.21.15.67:0/3726178642 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557d1e534c90 con 0x557d1e5cc8a0 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2c2000640 10 client.16314 handle_command_reply: tid=0 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 client.16314 shutdown 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 msgr2=0x557d1e5d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009e60 tx=0x7fc2cc009d80 comp rx=0 tx=0).stop 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 shutdown_connections 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557d1e5cc8a0 0x557d1e64c080 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557d1e5c0910 0x557d1e5cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d1e5bfff0 0x557d1e5d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 --2- 172.21.15.67:0/3726178642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1e5bde20 0x557d1e5d47f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 >> 172.21.15.67:0/3726178642 conn(0x557d1e5deae0 msgr2=0x557d1e5bcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2c2000640 0 client.16314 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 shutdown_connections 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 1 -- 172.21.15.67:0/3726178642 wait complete. 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.569+0000 7fc2ddcc4740 20 client.16314 trim_cache size 0 max 16384 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 56, 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "ee1bb40a": { 2024-09-10T23:17:53.571 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 56, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.0, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:53.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 msgr2=0x7fc2b80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 0x7fc2b80b7990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x557d1e5c3170 tx=0x7fc2c40e9980 comp rx=0 tx=0).stop 2024-09-10T23:17:53.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 msgr2=0x7fc2d419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d950 tx=0x7fc2c400de10 comp rx=0 tx=0).stop 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 shutdown_connections 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc2b80b54d0 0x7fc2b80b7990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d410f3c0 0x7fc2d41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2d4075f40 0x7fc2d419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 --2- 172.21.15.67:0/1345770983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d4075540 0x7fc2d419b410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.570+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 >> 172.21.15.67:0/1345770983 conn(0x7fc2d40fe810 msgr2=0x7fc2d40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.571+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 shutdown_connections 2024-09-10T23:17:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:53.571+0000 7fc2db200640 1 -- 172.21.15.67:0/1345770983 wait complete. 2024-09-10T23:17:54.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:53 smithi067 ceph-mon[30939]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 480 KiB/s wr, 7 op/s 2024-09-10T23:17:54.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:53 smithi092 ceph-mon[34223]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 480 KiB/s wr, 7 op/s 2024-09-10T23:17:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:53 smithi145 ceph-mon[34397]: pgmap v981: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 480 KiB/s wr, 7 op/s 2024-09-10T23:17:54.854 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee1bb40a', total seconds elapsed: 26.0 2024-09-10T23:17:54.854 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-10T23:17:54.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 -- 172.21.15.67:0/2484494194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 msgr2=0x7fb6c81051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2484494194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c81051f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009900 tx=0x7fb6b802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 -- 172.21.15.67:0/2484494194 shutdown_connections 2024-09-10T23:17:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2484494194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810b1a0 0x7fb6c81119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2484494194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810a820 0x7fb6c810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2484494194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c81051f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.026+0000 7fb6cca00640 1 -- 172.21.15.67:0/2484494194 >> 172.21.15.67:0/2484494194 conn(0x7fb6c8100840 msgr2=0x7fb6c8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.027+0000 7fb6cca00640 1 -- 172.21.15.67:0/2484494194 shutdown_connections 2024-09-10T23:17:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.027+0000 7fb6cca00640 1 -- 172.21.15.67:0/2484494194 wait complete. 2024-09-10T23:17:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 Processor -- start 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 -- start start 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810a820 0x7fb6c819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810b1a0 0x7fb6c81a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8114d10 con 0x7fb6c8104e10 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8114b90 con 0x7fb6c810a820 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6cca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8114e90 con 0x7fb6c810b1a0 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33126/0 (socket says 172.21.15.67:33126) 2024-09-10T23:17:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.028+0000 7fb6c7400640 1 -- 172.21.15.67:0/2545250057 learned_addr learned my addr 172.21.15.67:0/2545250057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c6a00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810a820 0x7fb6c819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7e00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810b1a0 0x7fb6c81a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 -- 172.21.15.67:0/2545250057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810b1a0 msgr2=0x7fb6c81a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810b1a0 0x7fb6c81a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 -- 172.21.15.67:0/2545250057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810a820 msgr2=0x7fb6c819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810a820 0x7fb6c819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 -- 172.21.15.67:0/2545250057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c81a1190 con 0x7fb6c8104e10 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.029+0000 7fb6c7400640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009230 tx=0x7fb6b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.030+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b803f030 con 0x7fb6c8104e10 2024-09-10T23:17:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.030+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81a1420 con 0x7fb6c8104e10 2024-09-10T23:17:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.030+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6b8003a80 con 0x7fb6c8104e10 2024-09-10T23:17:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.030+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81a7b40 con 0x7fb6c8104e10 2024-09-10T23:17:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.030+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8035070 con 0x7fb6c8104e10 2024-09-10T23:17:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.031+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c81051f0 con 0x7fb6c8104e10 2024-09-10T23:17:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.035+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb6b803ba20 con 0x7fb6c8104e10 2024-09-10T23:17:56.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.037+0000 7fb6bfe00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 0x7fb6ac0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:56.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.037+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7fb6b8107180 con 0x7fb6c8104e10 2024-09-10T23:17:56.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.037+0000 7fb6c6a00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 0x7fb6ac0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:56.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.038+0000 7fb6c6a00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 0x7fb6ac0b7a40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb6c810a4d0 tx=0x7fb6b4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:56.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.040+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6b80cd8c0 con 0x7fb6c8104e10 2024-09-10T23:17:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:55 smithi145 ceph-mon[34397]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 822 KiB/s wr, 6 op/s 2024-09-10T23:17:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:55 smithi092 ceph-mon[34223]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 822 KiB/s wr, 6 op/s 2024-09-10T23:17:56.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:55 smithi067 ceph-mon[30939]: pgmap v982: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 822 KiB/s wr, 6 op/s 2024-09-10T23:17:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.328+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c819c210 con 0x7fb6c8104e10 2024-09-10T23:17:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.328+0000 7fb6bfe00640 1 -- 172.21.15.67:0/2545250057 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fb6b80cd1c0 con 0x7fb6c8104e10 2024-09-10T23:17:56.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:17:56.329 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:17:56.330 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:17:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 msgr2=0x7fb6ac0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 0x7fb6ac0b7a40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb6c810a4d0 tx=0x7fb6b4005f90 comp rx=0 tx=0).stop 2024-09-10T23:17:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 msgr2=0x7fb6c819b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:56.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009230 tx=0x7fb6b8004270 comp rx=0 tx=0).stop 2024-09-10T23:17:56.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 shutdown_connections 2024-09-10T23:17:56.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb6ac0b5580 0x7fb6ac0b7a40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c810b1a0 0x7fb6c81a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c810a820 0x7fb6c819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 --2- 172.21.15.67:0/2545250057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6c8104e10 0x7fb6c819b0e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.331+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 >> 172.21.15.67:0/2545250057 conn(0x7fb6c8100840 msgr2=0x7fb6c81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.332+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 shutdown_connections 2024-09-10T23:17:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:56.332+0000 7fb6cca00640 1 -- 172.21.15.67:0/2545250057 wait complete. 2024-09-10T23:17:57.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:56 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2545250057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:57.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:56 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2545250057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:57.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:56 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2545250057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:57.286 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-10T23:17:57.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-10T23:17:57.783 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:58.111 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:57 smithi067 ceph-mon[30939]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 8 op/s 2024-09-10T23:17:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:57 smithi145 ceph-mon[34397]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 8 op/s 2024-09-10T23:17:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:57 smithi092 ceph-mon[34223]: pgmap v983: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 8 op/s 2024-09-10T23:17:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 -- 172.21.15.67:0/264889337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 msgr2=0x7efeec10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 --2- 172.21.15.67:0/264889337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec10f3d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efee4009ca0 tx=0x7efee402a8b0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 -- 172.21.15.67:0/264889337 shutdown_connections 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 --2- 172.21.15.67:0/264889337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec10f3d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 --2- 172.21.15.67:0/264889337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 0x7efeec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 --2- 172.21.15.67:0/264889337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 0x7efeec103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 -- 172.21.15.67:0/264889337 >> 172.21.15.67:0/264889337 conn(0x7efeec0fd5b0 msgr2=0x7efeec0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 -- 172.21.15.67:0/264889337 shutdown_connections 2024-09-10T23:17:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.412+0000 7efef2c00640 1 -- 172.21.15.67:0/264889337 wait complete. 2024-09-10T23:17:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.413+0000 7efef2c00640 1 Processor -- start 2024-09-10T23:17:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.413+0000 7efef2c00640 1 -- start start 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 0x7efeec19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 0x7efeec19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112b60 con 0x7efeec10cf70 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1129e0 con 0x7efeec104430 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112ce0 con 0x7efeec103b10 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33132/0 (socket says 172.21.15.67:33132) 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef2200640 1 -- 172.21.15.67:0/2831893340 learned_addr learned my addr 172.21.15.67:0/2831893340 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef1800640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 0x7efeec19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.414+0000 7efef0e00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 0x7efeec19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 -- 172.21.15.67:0/2831893340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 msgr2=0x7efeec19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 0x7efeec19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 -- 172.21.15.67:0/2831893340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 msgr2=0x7efeec19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 0x7efeec19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 -- 172.21.15.67:0/2831893340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec1a1260 con 0x7efeec10cf70 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efef2200640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efee4030750 tx=0x7efee4030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efeea000640 1 -- 172.21.15.67:0/2831893340 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4004040 con 0x7efeec10cf70 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.415+0000 7efeea000640 1 -- 172.21.15.67:0/2831893340 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efee4040070 con 0x7efeec10cf70 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.416+0000 7efeea000640 1 -- 172.21.15.67:0/2831893340 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4045aa0 con 0x7efeec10cf70 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.416+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec1a14f0 con 0x7efeec10cf70 2024-09-10T23:17:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.416+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec1a7c90 con 0x7efeec10cf70 2024-09-10T23:17:58.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.419+0000 7efef6be5740 1 Processor -- start 2024-09-10T23:17:58.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efeea000640 1 -- 172.21.15.67:0/2831893340 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efee40041e0 con 0x7efeec10cf70 2024-09-10T23:17:58.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 -- start start 2024-09-10T23:17:58.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558816051ca0 con 0x5588160dc2f0 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5588160410d0 con 0x5588160e8ba0 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.420+0000 7efef6be5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558816044ee0 con 0x5588160dcc10 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33142/0 (socket says 172.21.15.67:33142) 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 -- 172.21.15.67:0/3238136588 learned_addr learned my addr 172.21.15.67:0/3238136588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef1800640 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 -- 172.21.15.67:0/3238136588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 msgr2=0x5588160e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 -- 172.21.15.67:0/3238136588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 msgr2=0x5588160eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160eaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.421+0000 7efef0e00640 1 -- 172.21.15.67:0/3238136588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558815ff3400 con 0x5588160dc2f0 2024-09-10T23:17:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efef0e00640 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efed8009f90 tx=0x7efed8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2788cefe70296d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efeea000640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 0x7efed40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efeea000640 1 -- 172.21.15.67:0/2831893340 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7efee4107890 con 0x7efeec10cf70 2024-09-10T23:17:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efef1800640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 0x7efed40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efedea00640 1 -- 172.21.15.67:0/3238136588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8017610 con 0x5588160dc2f0 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efedea00640 1 -- 172.21.15.67:0/3238136588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed801c070 con 0x5588160dc2f0 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.422+0000 7efedea00640 1 -- 172.21.15.67:0/3238136588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8015840 con 0x5588160dc2f0 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 -- 172.21.15.67:0/3238136588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 msgr2=0x5588160dc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef1800640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 0x7efed40b7a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5588160db000 tx=0x7efee000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efed8009f90 tx=0x7efed8009560 comp rx=0 tx=0).stop 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 -- 172.21.15.67:0/3238136588 shutdown_connections 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 --2- 172.21.15.67:0/3238136588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160dc6d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.423+0000 7efef6be5740 1 -- 172.21.15.67:0/3238136588 >> 172.21.15.67:0/3238136588 conn(0x5588160da120 msgr2=0x5588160db2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.424+0000 7efef6be5740 1 -- 172.21.15.67:0/3238136588 shutdown_connections 2024-09-10T23:17:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.424+0000 7efef6be5740 1 -- 172.21.15.67:0/3238136588 wait complete. 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.424+0000 7efef6be5740 1 -- start start 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.424+0000 7efef6be5740 1 Processor -- start 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 0x5588160e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 0x5588160e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558815ea27a0 con 0x5588160dc2f0 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558815fe6a10 con 0x5588160dcc10 2024-09-10T23:17:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5588160eb7c0 con 0x5588160e8ba0 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef6be5740 10 client.0 _ll_register_callbacks cb 0x558815ff6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33144/0 (socket says 172.21.15.67:33144) 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef1800640 1 -- 172.21.15.67:0/3479889023 learned_addr learned my addr 172.21.15.67:0/3479889023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.425+0000 7efef0e00640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 0x5588160e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef2200640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 0x5588160e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 -- 172.21.15.67:0/3479889023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 msgr2=0x5588160e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 0x5588160e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 -- 172.21.15.67:0/3479889023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 msgr2=0x5588160e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 -- 172.21.15.67:0/3479889023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558815daeb90 con 0x5588160dcc10 2024-09-10T23:17:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef2200640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 0x5588160e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef1800640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efef0e00640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 0x5588160e58e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efed8004700 tx=0x7efed8009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48cbbdd94d46319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.426+0000 7efed3400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.427+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8017820 con 0x5588160dcc10 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.427+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed801c070 con 0x5588160dcc10 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.427+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7efed80179c0 con 0x5588160dcc10 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.428+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efed8031050 con 0x5588160dcc10 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.428+0000 7efef6be5740 10 client.36035 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.428+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5588160f71a0 con 0x5588160dcc10 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.428+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed801c070 con 0x5588160dcc10 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.428+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7efed802b4c0 con 0x5588160dcc10 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 10 client.36035 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 10 client.36035 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 20 client.36035 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5588160f8550 0x5588160fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 4 client.36035 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558816053bf0 con 0x5588160f8550 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.429+0000 7efef1800640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5588160f8550 0x5588160fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.430+0000 7efef1800640 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5588160f8550 0x5588160fa940 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b10c602682ce1a3 server_cookie=599a6fe06de59ae7 in_seq=0 out_seq=0 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.430+0000 7efed3400640 10 client.36035 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efed3400640 1 -- 172.21.15.67:0/3479889023 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558816053bf0 con 0x5588160f8550 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efed3400640 10 client.36035 handle_command_reply: tid=0 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 client.36035 shutdown 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 msgr2=0x5588160e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 0x5588160e58e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efed8004700 tx=0x7efed8009be0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 shutdown_connections 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5588160f8550 0x5588160fa940 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160e8ba0 0x5588160e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160dcc10 0x5588160e58e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 --2- 172.21.15.67:0/3479889023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efed3400640 0 client.36035 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 >> 172.21.15.67:0/3479889023 conn(0x5588160da120 msgr2=0x5588160dfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 shutdown_connections 2024-09-10T23:17:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.431+0000 7efef6be5740 1 -- 172.21.15.67:0/3479889023 wait complete. 2024-09-10T23:17:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.432+0000 7efef6be5740 20 client.36035 trim_cache size 0 max 16384 2024-09-10T23:17:58.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.454+0000 7efef6be5740 1 Processor -- start 2024-09-10T23:17:58.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.454+0000 7efef6be5740 1 -- start start 2024-09-10T23:17:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b46b0 con 0x5588160dc2f0 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b1f80 con 0x5588160e8ba0 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef6be5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1afdf0 con 0x5588160dcc10 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33152/0 (socket says 172.21.15.67:33152) 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef1800640 1 -- 172.21.15.67:0/1870074237 learned_addr learned my addr 172.21.15.67:0/1870074237 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef0e00640 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.455+0000 7efef2200640 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 -- 172.21.15.67:0/1870074237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 msgr2=0x5588160e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 -- 172.21.15.67:0/1870074237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 msgr2=0x5588160e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 -- 172.21.15.67:0/1870074237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588160e6240 con 0x5588160dc2f0 2024-09-10T23:17:58.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efef1800640 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efee0005880 tx=0x7efee0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a945204b4f0ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efede000640 1 -- 172.21.15.67:0/1870074237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee001ebe0 con 0x5588160dc2f0 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efede000640 1 -- 172.21.15.67:0/1870074237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efee001ed80 con 0x5588160dc2f0 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.456+0000 7efede000640 1 -- 172.21.15.67:0/1870074237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0004de0 con 0x5588160dc2f0 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.457+0000 7efef6be5740 1 -- 172.21.15.67:0/1870074237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 msgr2=0x5588160e47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.457+0000 7efef6be5740 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efee0005880 tx=0x7efee0019000 comp rx=0 tx=0).stop 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 -- 172.21.15.67:0/1870074237 shutdown_connections 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dcc10 0x5588160e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 --2- 172.21.15.67:0/1870074237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dc2f0 0x5588160e47d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 -- 172.21.15.67:0/1870074237 >> 172.21.15.67:0/1870074237 conn(0x5588160da120 msgr2=0x5588160e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 -- 172.21.15.67:0/1870074237 shutdown_connections 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 -- 172.21.15.67:0/1870074237 wait complete. 2024-09-10T23:17:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.458+0000 7efef6be5740 1 -- start start 2024-09-10T23:17:58.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 Processor -- start 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 0x5588160fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b0440 con 0x5588160dcc10 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b18b0 con 0x5588160e8ba0 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b0b10 con 0x5588160dc2f0 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef6be5740 10 client.0 _ll_register_callbacks cb 0x7efee0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33166/0 (socket says 172.21.15.67:33166) 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.459+0000 7efef0e00640 1 -- 172.21.15.67:0/3862338293 learned_addr learned my addr 172.21.15.67:0/3862338293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 0x5588160fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef2200640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 -- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 msgr2=0x5588160fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 -- 172.21.15.67:0/3862338293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 msgr2=0x5588160fc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef1800640 1 -- 172.21.15.67:0/3862338293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588161650d0 con 0x5588160dc2f0 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef0e00640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.460+0000 7efef2200640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efef1800640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 0x5588160fe2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efee000c010 tx=0x7efee0003bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3eebc01cd3a7666 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efed3400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0005100 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee001ed80 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9519+0+0 (secure 0 0 0) 0x7efee00052a0 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efee00054a0 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efef6be5740 10 client.26020 fetch_fsmap learned FSMap version 35 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.461+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5588161662b0 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efee001ed80 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7efee002a540 con 0x5588160dc2f0 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 10 client.26020 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:17:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 10 client.26020 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 20 client.26020 populate_metadata read hostname 'smithi067' 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558816166a70 0x558816168e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 4 client.26020 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.462+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x55881604f6a0 con 0x558816166a70 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.463+0000 7efef0e00640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558816166a70 0x558816168e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.463+0000 7efef0e00640 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558816166a70 0x558816168e90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0d9f31de690718 server_cookie=65b2ecb0f4d8aeba in_seq=0 out_seq=0 2024-09-10T23:17:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.463+0000 7efed3400640 10 client.26020 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.610+0000 7efed3400640 1 -- 172.21.15.67:0/3862338293 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55881604f6a0 con 0x558816166a70 2024-09-10T23:17:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.610+0000 7efed3400640 10 client.26020 handle_command_reply: tid=0 2024-09-10T23:17:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.610+0000 7efef6be5740 1 client.26020 shutdown 2024-09-10T23:17:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.610+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 msgr2=0x5588160fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.610+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 0x5588160fe2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efee000c010 tx=0x7efee0003bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 shutdown_connections 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558816166a70 0x558816168e90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588160e8ba0 0x5588160fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588160dcc10 0x5588160fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efed3400640 0 client.26020 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 --2- 172.21.15.67:0/3862338293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588160dc2f0 0x5588160fe2b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 >> 172.21.15.67:0/3862338293 conn(0x5588160da120 msgr2=0x5588160e7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 shutdown_connections 2024-09-10T23:17:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 1 -- 172.21.15.67:0/3862338293 wait complete. 2024-09-10T23:17:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.611+0000 7efef6be5740 20 client.26020 trim_cache size 0 max 16384 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 58, 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "e1dec70e": { 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 58, 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.0, 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:17:58.615 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 msgr2=0x7efed40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 0x7efed40b7a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5588160db000 tx=0x7efee000b390 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 msgr2=0x7efeec1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efee4030750 tx=0x7efee4030d50 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 shutdown_connections 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efed40b5580 0x7efed40b7a40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeec10cf70 0x7efeec1a0a90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec104430 0x7efeec19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 --2- 172.21.15.67:0/2831893340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efeec103b10 0x7efeec19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 >> 172.21.15.67:0/2831893340 conn(0x7efeec0fd5b0 msgr2=0x7efeec0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 shutdown_connections 2024-09-10T23:17:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:17:58.612+0000 7efef2c00640 1 -- 172.21.15.67:0/2831893340 wait complete. 2024-09-10T23:17:59.498 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e1dec70e', quiesce took 4.6 seconds. Will release after: 16.4 2024-09-10T23:18:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:17:59 smithi145 ceph-mon[34397]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-09-10T23:18:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:17:59 smithi092 ceph-mon[34223]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-09-10T23:18:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:17:59 smithi067 ceph-mon[30939]: pgmap v984: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-09-10T23:18:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:01 smithi145 ceph-mon[34397]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:18:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:01 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:18:02.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:01 smithi092 ceph-mon[34223]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:18:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:01 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:18:02.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:01 smithi067 ceph-mon[30939]: pgmap v985: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.5 MiB/s wr, 9 op/s 2024-09-10T23:18:02.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:01 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 22 op/s 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:18:03.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 22 op/s 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:18:03.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: pgmap v986: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 22 op/s 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:18:03.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:18:05.770 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:05 smithi092 ceph-mon[34223]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 24 op/s 2024-09-10T23:18:05.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:05 smithi067 ceph-mon[30939]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 24 op/s 2024-09-10T23:18:05.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:05 smithi145 ceph-mon[34397]: pgmap v987: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 MiB/s wr, 24 op/s 2024-09-10T23:18:07.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:07 smithi092 ceph-mon[34223]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 26 op/s 2024-09-10T23:18:07.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:07 smithi067 ceph-mon[30939]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 26 op/s 2024-09-10T23:18:07.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:07 smithi145 ceph-mon[34397]: pgmap v988: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 26 op/s 2024-09-10T23:18:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:09 smithi092 ceph-mon[34223]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 24 op/s 2024-09-10T23:18:09.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:09 smithi092 ceph-mon[34223]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T23:18:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:09 smithi067 ceph-mon[30939]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 24 op/s 2024-09-10T23:18:09.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:09 smithi067 ceph-mon[30939]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T23:18:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:09 smithi145 ceph-mon[34397]: pgmap v989: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 24 op/s 2024-09-10T23:18:09.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:09 smithi145 ceph-mon[34397]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T23:18:11.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:11 smithi092 ceph-mon[34223]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 874 KiB/s wr, 27 op/s 2024-09-10T23:18:11.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:11 smithi067 ceph-mon[30939]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 874 KiB/s wr, 27 op/s 2024-09-10T23:18:11.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:11 smithi145 ceph-mon[34397]: pgmap v991: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 874 KiB/s wr, 27 op/s 2024-09-10T23:18:13.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:13 smithi092 ceph-mon[34223]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 848 KiB/s wr, 10 op/s 2024-09-10T23:18:13.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:13 smithi067 ceph-mon[30939]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 848 KiB/s wr, 10 op/s 2024-09-10T23:18:13.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:13 smithi145 ceph-mon[34397]: pgmap v992: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 848 KiB/s wr, 10 op/s 2024-09-10T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:15 smithi092 ceph-mon[34223]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:18:15.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:15 smithi067 ceph-mon[30939]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:18:15.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:15 smithi145 ceph-mon[34397]: pgmap v993: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:18:15.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:18:16.280 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:18:16.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 -- 172.21.15.67:0/3182920585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 msgr2=0x7fd2bc115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 --2- 172.21.15.67:0/3182920585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc115d90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009ca0 tx=0x7fd2b402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 -- 172.21.15.67:0/3182920585 shutdown_connections 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 --2- 172.21.15.67:0/3182920585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc115d90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 --2- 172.21.15.67:0/3182920585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 0x7fd2bc113460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 --2- 172.21.15.67:0/3182920585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 0x7fd2bc077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 -- 172.21.15.67:0/3182920585 >> 172.21.15.67:0/3182920585 conn(0x7fd2bc1009e0 msgr2=0x7fd2bc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.986+0000 7fd2c4000640 1 -- 172.21.15.67:0/3182920585 shutdown_connections 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.987+0000 7fd2c4000640 1 -- 172.21.15.67:0/3182920585 wait complete. 2024-09-10T23:18:16.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.987+0000 7fd2c4000640 1 Processor -- start 2024-09-10T23:18:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 -- start start 2024-09-10T23:18:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 0x7fd2bc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 0x7fd2bc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc119270 con 0x7fd2bc1139a0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc1190f0 con 0x7fd2bc0776d0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c4000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc1193f0 con 0x7fd2bc077ff0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.988+0000 7fd2c2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34984/0 (socket says 172.21.15.67:34984) 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 -- 172.21.15.67:0/226100118 learned_addr learned my addr 172.21.15.67:0/226100118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c0c00640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 0x7fd2bc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c1600640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 0x7fd2bc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 -- 172.21.15.67:0/226100118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 msgr2=0x7fd2bc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 0x7fd2bc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 -- 172.21.15.67:0/226100118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 msgr2=0x7fd2bc19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 0x7fd2bc19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 -- 172.21.15.67:0/226100118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc1a57d0 con 0x7fd2bc1139a0 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.989+0000 7fd2c2000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4031040 tx=0x7fd2b4032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.990+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4032d80 con 0x7fd2bc1139a0 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.990+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2b40044d0 con 0x7fd2bc1139a0 2024-09-10T23:18:16.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.990+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc1a5a60 con 0x7fd2bc1139a0 2024-09-10T23:18:16.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.990+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4030ae0 con 0x7fd2bc1139a0 2024-09-10T23:18:16.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.990+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc1ac180 con 0x7fd2bc1139a0 2024-09-10T23:18:16.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.991+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc077ab0 con 0x7fd2bc1139a0 2024-09-10T23:18:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.993+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd2b4030c80 con 0x7fd2bc1139a0 2024-09-10T23:18:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.995+0000 7fd2ba000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 0x7fd2a40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.996+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9607+0+0 (secure 0 0 0) 0x7fd2b41075d0 con 0x7fd2bc1139a0 2024-09-10T23:18:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.996+0000 7fd2c1600640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 0x7fd2a40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:16.996+0000 7fd2c1600640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 0x7fd2a40b7910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000abe0 tx=0x7fd2b000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:17.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.000+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd2b40cdd10 con 0x7fd2bc1139a0 2024-09-10T23:18:17.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.242+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2bc1ac440 con 0x7fd2bc1139a0 2024-09-10T23:18:17.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.243+0000 7fd2ba000640 1 -- 172.21.15.67:0/226100118 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fd2b40cd610 con 0x7fd2bc1139a0 2024-09-10T23:18:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:18:17.245 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:18:17.245 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.245+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 msgr2=0x7fd2a40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.245+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 0x7fd2a40b7910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000abe0 tx=0x7fd2b000a2d0 comp rx=0 tx=0).stop 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.245+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 msgr2=0x7fd2bc1a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.245+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4031040 tx=0x7fd2b4032990 comp rx=0 tx=0).stop 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 shutdown_connections 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd2a40b5450 0x7fd2a40b7910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2bc1139a0 0x7fd2bc1a5000 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2bc077ff0 0x7fd2bc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:17.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 --2- 172.21.15.67:0/226100118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2bc0776d0 0x7fd2bc19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:17.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 >> 172.21.15.67:0/226100118 conn(0x7fd2bc1009e0 msgr2=0x7fd2bc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:17.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 shutdown_connections 2024-09-10T23:18:17.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:17.246+0000 7fd2c4000640 1 -- 172.21.15.67:0/226100118 wait complete. 2024-09-10T23:18:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:17 smithi067 ceph-mon[30939]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 819 KiB/s wr, 7 op/s 2024-09-10T23:18:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:17 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/226100118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:18:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:17 smithi145 ceph-mon[34397]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 819 KiB/s wr, 7 op/s 2024-09-10T23:18:17.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:17 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/226100118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:18:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:17 smithi092 ceph-mon[34223]: pgmap v994: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 819 KiB/s wr, 7 op/s 2024-09-10T23:18:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:17 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/226100118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:18:18.065 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id e1dec70e --release --await' 2024-09-10T23:18:18.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id e1dec70e --release --await 2024-09-10T23:18:18.444 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 -- 172.21.15.67:0/1802125038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 msgr2=0x7f9eb410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 --2- 172.21.15.67:0/1802125038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb410ac60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 -- 172.21.15.67:0/1802125038 shutdown_connections 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 --2- 172.21.15.67:0/1802125038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 0x7f9eb41119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 --2- 172.21.15.67:0/1802125038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb410ac60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 --2- 172.21.15.67:0/1802125038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 0x7f9eb41051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 -- 172.21.15.67:0/1802125038 >> 172.21.15.67:0/1802125038 conn(0x7f9eb4100840 msgr2=0x7f9eb4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 -- 172.21.15.67:0/1802125038 shutdown_connections 2024-09-10T23:18:19.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.039+0000 7f9eb9400640 1 -- 172.21.15.67:0/1802125038 wait complete. 2024-09-10T23:18:19.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.040+0000 7f9eb9400640 1 Processor -- start 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.040+0000 7f9eb9400640 1 -- start start 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 0x7f9eb419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 0x7f9eb41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4114d20 con 0x7f9eb410a820 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4114ba0 con 0x7f9eb410b1a0 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb9400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4114ea0 con 0x7f9eb4104e10 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55200/0 (socket says 172.21.15.67:55200) 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb3400640 1 -- 172.21.15.67:0/4082422271 learned_addr learned my addr 172.21.15.67:0/4082422271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:19.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 0x7f9eb419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.041+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 0x7f9eb41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 -- 172.21.15.67:0/4082422271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 msgr2=0x7f9eb419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 0x7f9eb419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 -- 172.21.15.67:0/4082422271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 msgr2=0x7f9eb41a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 0x7f9eb41a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 -- 172.21.15.67:0/4082422271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb41a1220 con 0x7f9eb410a820 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb3400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002a910 tx=0x7f9ea0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb0c00640 1 -- 172.21.15.67:0/4082422271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003f070 con 0x7f9eb410a820 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.042+0000 7f9eb0c00640 1 -- 172.21.15.67:0/4082422271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ea0003e40 con 0x7f9eb410a820 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.043+0000 7f9eb0c00640 1 -- 172.21.15.67:0/4082422271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0035030 con 0x7f9eb410a820 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.043+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb41a14b0 con 0x7f9eb410a820 2024-09-10T23:18:19.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.043+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb41a7bd0 con 0x7f9eb410a820 2024-09-10T23:18:19.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 Processor -- start 2024-09-10T23:18:19.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 -- start start 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9eb0c00640 1 -- 172.21.15.67:0/4082422271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9ea0031400 con 0x7f9eb410a820 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 0x5624653f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x562465402540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562465402a80 0x562465404e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56246536d290 con 0x5624653f6af0 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56246535c6c0 con 0x562465402a80 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.046+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5624653604d0 con 0x5624653f61d0 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x562465402540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x562465402540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55214/0 (socket says 172.21.15.67:55214) 2024-09-10T23:18:19.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb3e00640 1 -- 172.21.15.67:0/1064468482 learned_addr learned my addr 172.21.15.67:0/1064468482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:19.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 0x5624653f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb3400640 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562465402a80 0x562465404e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 -- 172.21.15.67:0/1064468482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562465402a80 msgr2=0x562465404e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562465402a80 0x562465404e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 -- 172.21.15.67:0/1064468482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 msgr2=0x562465402540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x562465402540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.047+0000 7f9eb8a00640 1 -- 172.21.15.67:0/1064468482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624652e9460 con 0x5624653f61d0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 0x5624653f65b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009f60 tx=0x7f9eac009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ef097af4f3fad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9ea9200640 1 -- 172.21.15.67:0/1064468482 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac016070 con 0x5624653f61d0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9eb0c00640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 0x7f9e980b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9ea9200640 1 -- 172.21.15.67:0/1064468482 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9eac004d20 con 0x5624653f61d0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9ea9200640 1 -- 172.21.15.67:0/1064468482 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0126b0 con 0x5624653f61d0 2024-09-10T23:18:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9eb0c00640 1 -- 172.21.15.67:0/4082422271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9519+0+0 (secure 0 0 0) 0x7f9ea0106df0 con 0x7f9eb410a820 2024-09-10T23:18:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.048+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 0x7f9e980b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/1064468482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 msgr2=0x5624653f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 0x5624653f65b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009f60 tx=0x7f9eac009f90 comp rx=0 tx=0).stop 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 0x7f9e980b7a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5624653f9350 tx=0x7f9ea400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/1064468482 shutdown_connections 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562465402a80 0x562465404e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x562465402540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/1064468482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f61d0 0x5624653f65b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/1064468482 >> 172.21.15.67:0/1064468482 conn(0x5624653f4000 msgr2=0x5624653f51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/1064468482 shutdown_connections 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.049+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/1064468482 wait complete. 2024-09-10T23:18:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 -- start start 2024-09-10T23:18:19.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 Processor -- start 2024-09-10T23:18:19.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x5624653ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624653fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5624651bdbb0 con 0x562465402a80 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5624652f76c0 con 0x5624653f61d0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562465405730 con 0x5624653f6af0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.050+0000 7f9ebd4a4740 10 client.0 _ll_register_callbacks cb 0x562465312490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55226/0 (socket says 172.21.15.67:55226) 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 -- 172.21.15.67:0/3762422940 learned_addr learned my addr 172.21.15.67:0/3762422940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624653fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x5624653ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 -- 172.21.15.67:0/3762422940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 msgr2=0x5624653fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624653fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 -- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 msgr2=0x5624653ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x5624653ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 -- 172.21.15.67:0/3762422940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624650bab00 con 0x562465402a80 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005ab0 tx=0x7f9eac004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a1c90c40a5d9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.051+0000 7f9e9d600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac029060 con 0x562465402a80 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9519+0+0 (secure 0 0 0) 0x7f9eac0051e0 con 0x562465402a80 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eac004d20 con 0x562465402a80 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9eac030050 con 0x562465402a80 2024-09-10T23:18:19.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9ebd4a4740 10 client.16356 fetch_fsmap learned FSMap version 35 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562465131960 con 0x562465402a80 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.052+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9eac004d20 con 0x562465402a80 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9eac00f040 con 0x562465402a80 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 10 client.16356 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 10 client.16356 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 20 client.16356 populate_metadata read hostname 'smithi067' 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5624654125b0 0x5624654149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 4 client.16356 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56246536f1e0 con 0x5624654125b0 2024-09-10T23:18:19.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.053+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5624654125b0 0x5624654149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.054+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5624654125b0 0x5624654149a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84857f939ee1ae9d server_cookie=fcdbdeca1a93db8b in_seq=0 out_seq=0 2024-09-10T23:18:19.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.054+0000 7f9e9d600640 10 client.16356 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9e9d600640 1 -- 172.21.15.67:0/3762422940 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56246536f1e0 con 0x5624654125b0 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9e9d600640 10 client.16356 handle_command_reply: tid=0 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 client.16356 shutdown 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 msgr2=0x5624654004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005ab0 tx=0x7f9eac004c70 comp rx=0 tx=0).stop 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 shutdown_connections 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5624654125b0 0x5624654149a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x5624654004d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624653fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3762422940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x5624653ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 >> 172.21.15.67:0/3762422940 conn(0x5624653f4000 msgr2=0x5624653f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9e9d600640 0 client.16356 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:18:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 shutdown_connections 2024-09-10T23:18:19.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3762422940 wait complete. 2024-09-10T23:18:19.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.055+0000 7f9ebd4a4740 20 client.16356 trim_cache size 0 max 16384 2024-09-10T23:18:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 Processor -- start 2024-09-10T23:18:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 -- start start 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 0x5624653fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41b4780 con 0x5624653f6af0 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41b2050 con 0x5624653f61d0 2024-09-10T23:18:19.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41afec0 con 0x562465402a80 2024-09-10T23:18:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9eb3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9eb3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55238/0 (socket says 172.21.15.67:55238) 2024-09-10T23:18:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.074+0000 7f9eb3400640 1 -- 172.21.15.67:0/3223427991 learned_addr learned my addr 172.21.15.67:0/3223427991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 0x5624653fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 -- 172.21.15.67:0/3223427991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 msgr2=0x5624653fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 0x5624653fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 -- 172.21.15.67:0/3223427991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 msgr2=0x56246540e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 -- 172.21.15.67:0/3223427991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624653f4a80 con 0x5624653f6af0 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 0x5624653fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9eb3400640 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ccbe0 tx=0x7f9ea0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46113ef1943314c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9e9fe00640 1 -- 172.21.15.67:0/3223427991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0049040 con 0x5624653f6af0 2024-09-10T23:18:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.075+0000 7f9e9fe00640 1 -- 172.21.15.67:0/3223427991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ea00cc9f0 con 0x5624653f6af0 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9e9fe00640 1 -- 172.21.15.67:0/3223427991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00d1910 con 0x5624653f6af0 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3223427991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 msgr2=0x5624653fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ccbe0 tx=0x7f9ea0005cb0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3223427991 shutdown_connections 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562465402a80 0x5624653fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5624653f6af0 0x5624653fcc60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/3223427991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3223427991 >> 172.21.15.67:0/3223427991 conn(0x5624653f4000 msgr2=0x56246540d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3223427991 shutdown_connections 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.076+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/3223427991 wait complete. 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 -- start start 2024-09-10T23:18:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 Processor -- start 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624654078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41b0510 con 0x562465402a80 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41b1980 con 0x5624653f61d0 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41b0be0 con 0x5624653f6af0 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.077+0000 7f9ebd4a4740 10 client.0 _ll_register_callbacks cb 0x7f9ea4004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55240/0 (socket says 172.21.15.67:55240) 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 -- 172.21.15.67:0/2482470576 learned_addr learned my addr 172.21.15.67:0/2482470576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 -- 172.21.15.67:0/2482470576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 msgr2=0x5624654078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb3400640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624654078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624654078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 -- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 msgr2=0x56246540b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 -- 172.21.15.67:0/2482470576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246547f180 con 0x562465402a80 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb3400640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624654078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9eb8a00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005980 tx=0x7f9eac009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad398e0386519663 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.078+0000 7f9e9d600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac01e020 con 0x562465402a80 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9519+0+0 (secure 0 0 0) 0x7f9eac0051e0 con 0x562465402a80 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00cc9f0 con 0x562465402a80 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9eac029040 con 0x562465402a80 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9ebd4a4740 10 client.16362 fetch_fsmap learned FSMap version 35 2024-09-10T23:18:19.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5624654803b0 con 0x562465402a80 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ea00cc9f0 con 0x562465402a80 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9eac0053e0 con 0x562465402a80 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9ebd4a4740 10 client.16362 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.079+0000 7f9ebd4a4740 10 client.16362 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.080+0000 7f9ebd4a4740 20 client.16362 populate_metadata read hostname 'smithi067' 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.080+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562465480b20 0x562465482f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.080+0000 7f9ebd4a4740 4 client.16362 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e1dec70e", "release": true, "await": true}] 2024-09-10T23:18:19.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.080+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e1dec70e", "release": true, "await": true}) -- 0x56246536ac90 con 0x562465480b20 2024-09-10T23:18:19.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.080+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562465480b20 0x562465482f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:18:19.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.081+0000 7f9eb3e00640 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562465480b20 0x562465482f40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b79598b0a0faa7 server_cookie=c074daa7edae40d1 in_seq=0 out_seq=0 2024-09-10T23:18:19.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.081+0000 7f9e9d600640 10 client.16362 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:18:19.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.107+0000 7f9e9d600640 1 -- 172.21.15.67:0/2482470576 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56246536ac90 con 0x562465480b20 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.107+0000 7f9e9d600640 10 client.16362 handle_command_reply: tid=0 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.107+0000 7f9ebd4a4740 1 client.16362 shutdown 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 msgr2=0x562465408030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005980 tx=0x7f9eac009db0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 shutdown_connections 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562465480b20 0x562465482f40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562465402a80 0x562465408030 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624653f6af0 0x5624654078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 --2- 172.21.15.67:0/2482470576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624653f61d0 0x56246540b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9e9d600640 0 client.16362 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 >> 172.21.15.67:0/2482470576 conn(0x5624653f4000 msgr2=0x5624653feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 shutdown_connections 2024-09-10T23:18:19.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 1 -- 172.21.15.67:0/2482470576 wait complete. 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.108+0000 7f9ebd4a4740 20 client.16362 trim_cache size 0 max 16384 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 60, 2024-09-10T23:18:19.111 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "e1dec70e": { 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 60, 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 141.0, 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:18:19.112 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:18:19.113 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 msgr2=0x7f9e980b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 0x7f9e980b7a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5624653f9350 tx=0x7f9ea400b390 comp rx=0 tx=0).stop 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 msgr2=0x7f9eb419b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002a910 tx=0x7f9ea0003c20 comp rx=0 tx=0).stop 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 shutdown_connections 2024-09-10T23:18:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9e980b5580 0x7f9e980b7a40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb410b1a0 0x7f9eb41a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb410a820 0x7f9eb419b6b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 --2- 172.21.15.67:0/4082422271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9eb4104e10 0x7f9eb419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 >> 172.21.15.67:0/4082422271 conn(0x7f9eb4100840 msgr2=0x7f9eb4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 shutdown_connections 2024-09-10T23:18:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:18:19.109+0000 7f9eb9400640 1 -- 172.21.15.67:0/4082422271 wait complete. 2024-09-10T23:18:19.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:19 smithi145 ceph-mon[34397]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 410 KiB/s wr, 7 op/s 2024-09-10T23:18:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:19 smithi145 ceph-mon[34397]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T23:18:19.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:19 smithi067 ceph-mon[30939]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 410 KiB/s wr, 7 op/s 2024-09-10T23:18:19.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:19 smithi067 ceph-mon[30939]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:19 smithi092 ceph-mon[34223]: pgmap v995: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 410 KiB/s wr, 7 op/s 2024-09-10T23:18:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:19 smithi092 ceph-mon[34223]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T23:18:20.456 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e1dec70e', total seconds elapsed: 25.6 2024-09-10T23:18:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:21 smithi092 ceph-mon[34223]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 410 KiB/s wr, 6 op/s 2024-09-10T23:18:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:21 smithi067 ceph-mon[30939]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 410 KiB/s wr, 6 op/s 2024-09-10T23:18:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:21 smithi145 ceph-mon[34397]: pgmap v997: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 410 KiB/s wr, 6 op/s 2024-09-10T23:18:24.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:23 smithi092 ceph-mon[34223]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 819 KiB/s wr, 6 op/s 2024-09-10T23:18:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:23 smithi067 ceph-mon[30939]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 819 KiB/s wr, 6 op/s 2024-09-10T23:18:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:23 smithi145 ceph-mon[34397]: pgmap v998: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 819 KiB/s wr, 6 op/s 2024-09-10T23:18:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:25 smithi092 ceph-mon[34223]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:18:26.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:25 smithi067 ceph-mon[30939]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:18:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:25 smithi145 ceph-mon[34397]: pgmap v999: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-09-10T23:18:28.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:27 smithi092 ceph-mon[34223]: pgmap v1000: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-09-10T23:18:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:27 smithi067 ceph-mon[30939]: pgmap v1000: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-09-10T23:18:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:27 smithi145 ceph-mon[34397]: pgmap v1000: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-09-10T23:18:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:29 smithi092 ceph-mon[34223]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-09-10T23:18:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:29 smithi067 ceph-mon[30939]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-09-10T23:18:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:29 smithi145 ceph-mon[34397]: pgmap v1001: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-09-10T23:18:32.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:31 smithi092 ceph-mon[34223]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:18:32.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:31 smithi067 ceph-mon[30939]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:18:32.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:31 smithi145 ceph-mon[34397]: pgmap v1002: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-09-10T23:18:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:33 smithi092 ceph-mon[34223]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:18:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:33 smithi067 ceph-mon[30939]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:18:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:33 smithi145 ceph-mon[34397]: pgmap v1003: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-09-10T23:18:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:35 smithi092 ceph-mon[34223]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:35 smithi067 ceph-mon[30939]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:36.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:35 smithi145 ceph-mon[34397]: pgmap v1004: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:37 smithi092 ceph-mon[34223]: pgmap v1005: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:18:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:37 smithi067 ceph-mon[30939]: pgmap v1005: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:18:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:37 smithi145 ceph-mon[34397]: pgmap v1005: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:18:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:39 smithi092 ceph-mon[34223]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:39 smithi067 ceph-mon[30939]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:39 smithi145 ceph-mon[34397]: pgmap v1006: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:18:41.932 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:41 smithi145 ceph-mon[34397]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:41 smithi092 ceph-mon[34223]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T23:18:42.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:41 smithi067 ceph-mon[30939]: pgmap v1007: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T23:18:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:43 smithi092 ceph-mon[34223]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:44.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:43 smithi067 ceph-mon[30939]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:44.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:43 smithi145 ceph-mon[34397]: pgmap v1008: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:46.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:45 smithi067 ceph-mon[30939]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:18:46.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:45 smithi092 ceph-mon[34223]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:18:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:45 smithi145 ceph-mon[34397]: pgmap v1009: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:18:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:47 smithi092 ceph-mon[34223]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:18:48.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:47 smithi067 ceph-mon[30939]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:18:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:47 smithi145 ceph-mon[34397]: pgmap v1010: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:18:50.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:49 smithi092 ceph-mon[34223]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:18:50.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:49 smithi067 ceph-mon[30939]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:18:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:49 smithi145 ceph-mon[34397]: pgmap v1011: 129 pgs: 129 active+clean; 10 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:18:52.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:51 smithi092 ceph-mon[34223]: pgmap v1012: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:18:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:51 smithi067 ceph-mon[30939]: pgmap v1012: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:18:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:51 smithi145 ceph-mon[34397]: pgmap v1012: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:18:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:53 smithi092 ceph-mon[34223]: pgmap v1013: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:54.207 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:53 smithi145 ceph-mon[34397]: pgmap v1013: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:54.207 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:53 smithi067 ceph-mon[30939]: pgmap v1013: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:18:56.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:55 smithi092 ceph-mon[34223]: pgmap v1014: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:18:56.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:55 smithi067 ceph-mon[30939]: pgmap v1014: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:18:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:55 smithi145 ceph-mon[34397]: pgmap v1014: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:18:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:57 smithi092 ceph-mon[34223]: pgmap v1015: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:18:58.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:57 smithi067 ceph-mon[30939]: pgmap v1015: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:18:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:57 smithi145 ceph-mon[34397]: pgmap v1015: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:19:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:18:59 smithi092 ceph-mon[34223]: pgmap v1016: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:19:00.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:18:59 smithi067 ceph-mon[30939]: pgmap v1016: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:19:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:18:59 smithi145 ceph-mon[34397]: pgmap v1016: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 12 op/s 2024-09-10T23:19:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:01 smithi092 ceph-mon[34223]: pgmap v1017: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-10T23:19:02.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:01 smithi067 ceph-mon[30939]: pgmap v1017: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-10T23:19:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:01 smithi145 ceph-mon[34397]: pgmap v1017: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: pgmap v1018: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: pgmap v1018: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: pgmap v1018: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 27 op/s 2024-09-10T23:19:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T23:19:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:19:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:04.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:05.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:06.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:05 smithi092 ceph-mon[34223]: pgmap v1020: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:19:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:05 smithi067 ceph-mon[30939]: pgmap v1020: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:19:06.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:05 smithi145 ceph-mon[34397]: pgmap v1020: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 33 op/s 2024-09-10T23:19:07.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:06 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:06 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:06 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: pgmap v1021: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: pgmap v1021: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: pgmap v1021: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: pgmap v1022: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 34 op/s 2024-09-10T23:19:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:19:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:19:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:09 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: pgmap v1022: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 34 op/s 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:19:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:09 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: pgmap v1022: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 34 op/s 2024-09-10T23:19:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:19:10.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:19:10.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:09 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:19:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:11 smithi092 ceph-mon[34223]: pgmap v1023: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:12.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:11 smithi067 ceph-mon[30939]: pgmap v1023: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:12.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:11 smithi145 ceph-mon[34397]: pgmap v1023: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 34 op/s 2024-09-10T23:19:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:13 smithi092 ceph-mon[34223]: pgmap v1024: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:19:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:13 smithi092 ceph-mon[34223]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T23:19:14.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:13 smithi067 ceph-mon[30939]: pgmap v1024: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:19:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:13 smithi067 ceph-mon[30939]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T23:19:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:13 smithi145 ceph-mon[34397]: pgmap v1024: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T23:19:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:13 smithi145 ceph-mon[34397]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T23:19:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:15 smithi092 ceph-mon[34223]: pgmap v1026: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:16.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:15 smithi067 ceph-mon[30939]: pgmap v1026: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:16.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:15 smithi145 ceph-mon[34397]: pgmap v1026: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:17 smithi092 ceph-mon[34223]: pgmap v1027: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:19:18.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:17 smithi067 ceph-mon[30939]: pgmap v1027: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:19:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:17 smithi145 ceph-mon[34397]: pgmap v1027: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:19:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:19 smithi092 ceph-mon[34223]: pgmap v1028: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:19:20.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:19 smithi067 ceph-mon[30939]: pgmap v1028: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:19:20.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:19 smithi145 ceph-mon[34397]: pgmap v1028: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:19:22.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:21 smithi092 ceph-mon[34223]: pgmap v1029: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:19:22.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:21 smithi067 ceph-mon[30939]: pgmap v1029: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:19:22.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:21 smithi145 ceph-mon[34397]: pgmap v1029: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:19:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:23 smithi092 ceph-mon[34223]: pgmap v1030: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:24.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:23 smithi067 ceph-mon[30939]: pgmap v1030: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:23 smithi145 ceph-mon[34397]: pgmap v1030: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:19:26.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:25 smithi092 ceph-mon[34223]: pgmap v1031: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:19:26.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:25 smithi067 ceph-mon[30939]: pgmap v1031: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:19:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:25 smithi145 ceph-mon[34397]: pgmap v1031: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:19:28.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:27 smithi092 ceph-mon[34223]: pgmap v1032: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:19:28.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:27 smithi067 ceph-mon[30939]: pgmap v1032: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:19:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:27 smithi145 ceph-mon[34397]: pgmap v1032: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:19:30.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:29 smithi092 ceph-mon[34223]: pgmap v1033: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:19:30.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:29 smithi067 ceph-mon[30939]: pgmap v1033: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:19:30.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:29 smithi145 ceph-mon[34397]: pgmap v1033: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:19:32.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:31 smithi092 ceph-mon[34223]: pgmap v1034: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:32.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:31 smithi067 ceph-mon[30939]: pgmap v1034: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:32.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:31 smithi145 ceph-mon[34397]: pgmap v1034: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:34.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:33 smithi092 ceph-mon[34223]: pgmap v1035: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:19:34.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:33 smithi067 ceph-mon[30939]: pgmap v1035: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:19:34.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:33 smithi145 ceph-mon[34397]: pgmap v1035: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:19:36.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:35 smithi092 ceph-mon[34223]: pgmap v1036: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:19:36.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:35 smithi067 ceph-mon[30939]: pgmap v1036: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:19:36.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:35 smithi145 ceph-mon[34397]: pgmap v1036: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-09-10T23:19:38.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:37 smithi092 ceph-mon[34223]: pgmap v1037: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:19:38.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:37 smithi067 ceph-mon[30939]: pgmap v1037: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:19:38.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:37 smithi145 ceph-mon[34397]: pgmap v1037: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:19:38.457 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:19:38.457 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:19:38.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:19:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 -- 172.21.15.67:0/2141467357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 msgr2=0x7f3ed81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 --2- 172.21.15.67:0/2141467357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81119a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ca0 tx=0x7f3ed002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 -- 172.21.15.67:0/2141467357 shutdown_connections 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 --2- 172.21.15.67:0/2141467357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81119a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 --2- 172.21.15.67:0/2141467357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 0x7f3ed810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 --2- 172.21.15.67:0/2141467357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 -- 172.21.15.67:0/2141467357 >> 172.21.15.67:0/2141467357 conn(0x7f3ed8100840 msgr2=0x7f3ed8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 -- 172.21.15.67:0/2141467357 shutdown_connections 2024-09-10T23:19:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.797+0000 7f3edee00640 1 -- 172.21.15.67:0/2141467357 wait complete. 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.798+0000 7f3edee00640 1 Processor -- start 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.798+0000 7f3edee00640 1 -- start start 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 0x7f3ed819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8114d20 con 0x7f3ed810b1a0 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8114ba0 con 0x7f3ed8104e10 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8114ea0 con 0x7f3ed810a820 2024-09-10T23:19:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edda00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3ede400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edda00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:33390/0 (socket says 172.21.15.67:33390) 2024-09-10T23:19:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edda00640 1 -- 172.21.15.67:0/3650889753 learned_addr learned my addr 172.21.15.67:0/3650889753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.799+0000 7f3edd000640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 0x7f3ed819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 -- 172.21.15.67:0/3650889753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 msgr2=0x7f3ed819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 0x7f3ed819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 -- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 msgr2=0x7f3ed81a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 -- 172.21.15.67:0/3650889753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a1200 con 0x7f3ed8104e10 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.800+0000 7f3edda00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e9f0 tx=0x7f3ecc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.801+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00cd80 con 0x7f3ed8104e10 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.801+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a14f0 con 0x7f3ed8104e10 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.801+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ecc0108c0 con 0x7f3ed8104e10 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.801+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0187c0 con 0x7f3ed8104e10 2024-09-10T23:19:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.801+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a7d80 con 0x7f3ed8104e10 2024-09-10T23:19:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.802+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed81051f0 con 0x7f3ed8104e10 2024-09-10T23:19:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.804+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3ecc010430 con 0x7f3ed8104e10 2024-09-10T23:19:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.806+0000 7f3ed6000640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 0x7f3ec00b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.806+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9519+0+0 (secure 0 0 0) 0x7f3ecc0e67b0 con 0x7f3ed8104e10 2024-09-10T23:19:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.806+0000 7f3edd000640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 0x7f3ec00b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.807+0000 7f3edd000640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 0x7f3ec00b79c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8103120 tx=0x7f3ec4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:39.807+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3ecc0ac8a0 con 0x7f3ed8104e10 2024-09-10T23:19:40.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.076+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed819c280 con 0x7f3ed8104e10 2024-09-10T23:19:40.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.077+0000 7f3ed6000640 1 -- 172.21.15.67:0/3650889753 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7f3ed819c280 con 0x7f3ed8104e10 2024-09-10T23:19:40.078 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:19:40.079 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:19:40.079 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:19:40.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 msgr2=0x7f3ec00b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:40.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 0x7f3ec00b79c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8103120 tx=0x7f3ec4005f90 comp rx=0 tx=0).stop 2024-09-10T23:19:40.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 msgr2=0x7f3ed819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:40.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e9f0 tx=0x7f3ecc00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:19:40.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 shutdown_connections 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f3ec00b5500 0x7f3ec00b79c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed810b1a0 0x7f3ed81a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed810a820 0x7f3ed819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 --2- 172.21.15.67:0/3650889753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ed8104e10 0x7f3ed819b150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 >> 172.21.15.67:0/3650889753 conn(0x7f3ed8100840 msgr2=0x7f3ed8108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 shutdown_connections 2024-09-10T23:19:40.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:40.080+0000 7f3edee00640 1 -- 172.21.15.67:0/3650889753 wait complete. 2024-09-10T23:19:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:39 smithi092 ceph-mon[34223]: pgmap v1038: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:40.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:39 smithi067 ceph-mon[30939]: pgmap v1038: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:39 smithi145 ceph-mon[34397]: pgmap v1038: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:41.024 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:19:41.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:19:41.079 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:40 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3650889753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:19:41.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:40 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3650889753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:19:41.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:40 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3650889753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:19:41.614 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:19:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:41 smithi067 ceph-mon[30939]: pgmap v1039: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:42.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:41 smithi092 ceph-mon[34223]: pgmap v1039: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:42.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:41 smithi145 ceph-mon[34397]: pgmap v1039: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:19:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:43 smithi092 ceph-mon[34223]: pgmap v1040: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:19:44.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:43 smithi067 ceph-mon[30939]: pgmap v1040: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:19:44.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:43 smithi145 ceph-mon[34397]: pgmap v1040: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 12 op/s 2024-09-10T23:19:46.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:45 smithi067 ceph-mon[30939]: pgmap v1041: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:19:46.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:45 smithi092 ceph-mon[34223]: pgmap v1041: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:19:46.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:45 smithi145 ceph-mon[34397]: pgmap v1041: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:19:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.713+0000 7f7912e00640 1 -- 172.21.15.67:0/3446190785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 msgr2=0x7f790c102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.713+0000 7f7912e00640 1 --2- 172.21.15.67:0/3446190785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c102ff0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7900009900 tx=0x7f790002a410 comp rx=0 tx=0).stop 2024-09-10T23:19:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 -- 172.21.15.67:0/3446190785 shutdown_connections 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 --2- 172.21.15.67:0/3446190785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 --2- 172.21.15.67:0/3446190785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 0x7f790c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 --2- 172.21.15.67:0/3446190785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c102ff0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 -- 172.21.15.67:0/3446190785 >> 172.21.15.67:0/3446190785 conn(0x7f790c0fe660 msgr2=0x7f790c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 -- 172.21.15.67:0/3446190785 shutdown_connections 2024-09-10T23:19:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.714+0000 7f7912e00640 1 -- 172.21.15.67:0/3446190785 wait complete. 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 Processor -- start 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 -- start start 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 0x7f790c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112b10 con 0x7f790c102c10 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112990 con 0x7f790c108fa0 2024-09-10T23:19:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.715+0000 7f7912e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112c90 con 0x7f790c108620 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47474/0 (socket says 172.21.15.67:47474) 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 -- 172.21.15.67:0/335527628 learned_addr learned my addr 172.21.15.67:0/335527628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7912400640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911000640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 0x7f790c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 -- 172.21.15.67:0/335527628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 msgr2=0x7f790c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 0x7f790c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 -- 172.21.15.67:0/335527628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 msgr2=0x7f790c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7911a00640 1 -- 172.21.15.67:0/335527628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c1a73a0 con 0x7f790c102c10 2024-09-10T23:19:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.716+0000 7f7912400640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.717+0000 7f7911a00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7900009230 tx=0x7f7900004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.717+0000 7f790a000640 1 -- 172.21.15.67:0/335527628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790003f030 con 0x7f790c102c10 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.717+0000 7f790a000640 1 -- 172.21.15.67:0/335527628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7900003a80 con 0x7f790c102c10 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.717+0000 7f790a000640 1 -- 172.21.15.67:0/335527628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900035070 con 0x7f790c102c10 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.717+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c1a7630 con 0x7f790c102c10 2024-09-10T23:19:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.718+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1a7ac0 con 0x7f790c102c10 2024-09-10T23:19:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.722+0000 7f790a000640 1 -- 172.21.15.67:0/335527628 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f79000035f0 con 0x7f790c102c10 2024-09-10T23:19:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.724+0000 7f790a000640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 0x7f78f40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.724+0000 7f790a000640 1 -- 172.21.15.67:0/335527628 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9519+0+0 (secure 0 0 0) 0x7f7900106bc0 con 0x7f790c102c10 2024-09-10T23:19:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.724+0000 7f7911000640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 0x7f78f40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.725+0000 7f7911000640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 0x7f78f40b79c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f790c076f90 tx=0x7f78f800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 Processor -- start 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 -- start start 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e9e2541ca0 con 0x55e9e25ccc10 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e9e25310d0 con 0x55e9e25d8ba0 2024-09-10T23:19:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.773+0000 7f7916e2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9e2534ee0 con 0x55e9e25cc2f0 2024-09-10T23:19:46.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47490/0 (socket says 172.21.15.67:47490) 2024-09-10T23:19:46.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 -- 172.21.15.67:0/1619602987 learned_addr learned my addr 172.21.15.67:0/1619602987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7912400640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911a00640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 -- 172.21.15.67:0/1619602987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 msgr2=0x55e9e25cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 -- 172.21.15.67:0/1619602987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 msgr2=0x55e9e25daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7911000640 1 -- 172.21.15.67:0/1619602987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e24e3400 con 0x55e9e25ccc10 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.774+0000 7f7912400640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.775+0000 7f7911000640 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f78f8003c70 tx=0x7f78f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af2f1203b9ec932 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.775+0000 7f78fea00640 1 -- 172.21.15.67:0/1619602987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8003ea0 con 0x55e9e25ccc10 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.775+0000 7f78fea00640 1 -- 172.21.15.67:0/1619602987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f801e8b0 con 0x55e9e25ccc10 2024-09-10T23:19:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.775+0000 7f78fea00640 1 -- 172.21.15.67:0/1619602987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80268d0 con 0x55e9e25ccc10 2024-09-10T23:19:46.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 -- 172.21.15.67:0/1619602987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 msgr2=0x55e9e25d8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f78f8003c70 tx=0x7f78f8013000 comp rx=0 tx=0).stop 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 -- 172.21.15.67:0/1619602987 shutdown_connections 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d8660 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1619602987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 -- 172.21.15.67:0/1619602987 >> 172.21.15.67:0/1619602987 conn(0x55e9e25ca120 msgr2=0x55e9e25cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 -- 172.21.15.67:0/1619602987 shutdown_connections 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.776+0000 7f7916e2c740 1 -- 172.21.15.67:0/1619602987 wait complete. 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.777+0000 7f7916e2c740 1 -- start start 2024-09-10T23:19:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.777+0000 7f7916e2c740 1 Processor -- start 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.777+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e9e23927a0 con 0x55e9e25ccc10 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e9e24d6a10 con 0x55e9e25d8ba0 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9e25db820 con 0x55e9e25cc2f0 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7916e2c740 10 client.0 _ll_register_callbacks cb 0x55e9e24e6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47492/0 (socket says 172.21.15.67:47492) 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911000640 1 -- 172.21.15.67:0/1092026946 learned_addr learned my addr 172.21.15.67:0/1092026946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7912400640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 -- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 msgr2=0x55e9e25d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 -- 172.21.15.67:0/1092026946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 msgr2=0x55e9e25d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.778+0000 7f7911a00640 1 -- 172.21.15.67:0/1092026946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e229eb90 con 0x55e9e25cc2f0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f7911000640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f7911a00640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25d84b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7900103ed0 tx=0x7f7900104380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3367eb4b09ac949 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f78f3400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000d1020 con 0x55e9e25cc2f0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9519+0+0 (secure 0 0 0) 0x7f79000cd110 con 0x55e9e25cc2f0 2024-09-10T23:19:46.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f801e8b0 con 0x55e9e25cc2f0 2024-09-10T23:19:46.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7900049040 con 0x55e9e25cc2f0 2024-09-10T23:19:46.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f7916e2c740 10 client.26056 fetch_fsmap learned FSMap version 35 2024-09-10T23:19:46.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.779+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9e25e7180 con 0x55e9e25cc2f0 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f801e8b0 con 0x55e9e25cc2f0 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f79000d5d00 con 0x55e9e25cc2f0 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 10 client.26056 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 10 client.26056 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 20 client.26056 populate_metadata read hostname 'smithi067' 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e25e8570 0x55e9e25ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 4 client.26056 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.780+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e2543bf0 con 0x55e9e25e8570 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.781+0000 7f7911000640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e25e8570 0x55e9e25ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.782+0000 7f7911000640 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e25e8570 0x55e9e25ea9a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335da2e2436bb4f6 server_cookie=78b4912b4a7cfd22 in_seq=0 out_seq=0 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.782+0000 7f78f3400640 10 client.26056 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.782+0000 7f78f3400640 1 -- 172.21.15.67:0/1092026946 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9e2543bf0 con 0x55e9e25e8570 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.782+0000 7f78f3400640 10 client.26056 handle_command_reply: tid=0 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 client.26056 shutdown 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 msgr2=0x55e9e25d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25d84b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7900103ed0 tx=0x7f7900104380 comp rx=0 tx=0).stop 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 shutdown_connections 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e25e8570 0x55e9e25ea9a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1092026946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25cc2f0 0x55e9e25d84b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f78f3400640 0 client.26056 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 >> 172.21.15.67:0/1092026946 conn(0x55e9e25ca120 msgr2=0x55e9e25cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 shutdown_connections 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 1 -- 172.21.15.67:0/1092026946 wait complete. 2024-09-10T23:19:46.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.783+0000 7f7916e2c740 20 client.26056 trim_cache size 0 max 16384 2024-09-10T23:19:46.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 Processor -- start 2024-09-10T23:19:46.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 -- start start 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 0x55e9e25d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b4750 con 0x55e9e25d8ba0 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b2020 con 0x55e9e25cc2f0 2024-09-10T23:19:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.802+0000 7f7916e2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1afe90 con 0x55e9e25ccc10 2024-09-10T23:19:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:33440/0 (socket says 172.21.15.67:33440) 2024-09-10T23:19:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 -- 172.21.15.67:0/1591987125 learned_addr learned my addr 172.21.15.67:0/1591987125 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 0x55e9e25d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7912400640 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 -- 172.21.15.67:0/1591987125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 msgr2=0x55e9e25d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 0x55e9e25d4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 -- 172.21.15.67:0/1591987125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25d8ba0 msgr2=0x55e9e25d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911a00640 1 -- 172.21.15.67:0/1591987125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e25c5960 con 0x55e9e25cc2f0 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.803+0000 7f7911000640 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 0x55e9e25d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.804+0000 7f7911a00640 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7900005c90 tx=0x7f7900009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78041629c887a97e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.804+0000 7f78fe000640 1 -- 172.21.15.67:0/1591987125 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000d1040 con 0x55e9e25cc2f0 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.804+0000 7f78fe000640 1 -- 172.21.15.67:0/1591987125 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79000cc7c0 con 0x55e9e25cc2f0 2024-09-10T23:19:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.804+0000 7f78fe000640 1 -- 172.21.15.67:0/1591987125 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000cd170 con 0x55e9e25cc2f0 2024-09-10T23:19:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 -- 172.21.15.67:0/1591987125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 msgr2=0x55e9e25c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7900005c90 tx=0x7f7900009260 comp rx=0 tx=0).stop 2024-09-10T23:19:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 -- 172.21.15.67:0/1591987125 shutdown_connections 2024-09-10T23:19:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25d8ba0 0x55e9e25d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25ccc10 0x55e9e25d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 --2- 172.21.15.67:0/1591987125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25c9af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 -- 172.21.15.67:0/1591987125 >> 172.21.15.67:0/1591987125 conn(0x55e9e25ca120 msgr2=0x55e9e25d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.805+0000 7f7916e2c740 1 -- 172.21.15.67:0/1591987125 shutdown_connections 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.806+0000 7f7916e2c740 1 -- 172.21.15.67:0/1591987125 wait complete. 2024-09-10T23:19:46.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.806+0000 7f7916e2c740 1 -- start start 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.806+0000 7f7916e2c740 1 Processor -- start 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25d8ba0 0x55e9e25dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b04e0 con 0x55e9e25ccc10 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b1950 con 0x55e9e25cc2f0 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b0bb0 con 0x55e9e25d8ba0 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.807+0000 7f7916e2c740 10 client.0 _ll_register_callbacks cb 0x7f79000c8b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:19:46.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25e0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:33452/0 (socket says 172.21.15.67:33452) 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911a00640 1 -- 172.21.15.67:0/2157703511 learned_addr learned my addr 172.21.15.67:0/2157703511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47520/0 (socket says 172.21.15.67:47520) 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7912400640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25d8ba0 0x55e9e25dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 -- 172.21.15.67:0/2157703511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25d8ba0 msgr2=0x55e9e25dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25d8ba0 0x55e9e25dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 -- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 msgr2=0x55e9e25e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:19:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.808+0000 7f7911000640 1 -- 172.21.15.67:0/2157703511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e25dbfb0 con 0x55e9e25ccc10 2024-09-10T23:19:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.809+0000 7f7911000640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55e9e25eb210 tx=0x7f78f8026490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fd77c119903046 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:19:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.809+0000 7f78f3400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:19:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.809+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005040 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.809+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9519+0+0 (secure 0 0 0) 0x7f78f80051e0 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.809+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79000cc7c0 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.810+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78f80053e0 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.810+0000 7f7916e2c740 10 client.16386 fetch_fsmap learned FSMap version 35 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.810+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9e265e360 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.810+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79000cc7c0 con 0x55e9e25ccc10 2024-09-10T23:19:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f78f8026580 con 0x55e9e25ccc10 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 10 client.16386 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 10 client.16386 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 20 client.16386 populate_metadata read hostname 'smithi067' 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e265e850 0x55e9e2660c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 4 client.16386 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55e9e253f6a0 con 0x55e9e265e850 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.811+0000 7f7911a00640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e265e850 0x55e9e2660c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:19:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.812+0000 7f78f3400640 10 client.16386 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:19:46.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.812+0000 7f7911a00640 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e265e850 0x55e9e2660c40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6589e9a0f06fa6 server_cookie=45b09a0583dea42c in_seq=0 out_seq=0 2024-09-10T23:19:46.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f78f3400640 1 -- 172.21.15.67:0/2157703511 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e9e253f6a0 con 0x55e9e265e850 2024-09-10T23:19:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f78f3400640 10 client.16386 handle_command_reply: tid=0 2024-09-10T23:19:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 client.16386 shutdown 2024-09-10T23:19:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 msgr2=0x55e9e25e1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55e9e25eb210 tx=0x7f78f8026490 comp rx=0 tx=0).stop 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 shutdown_connections 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55e9e265e850 0x55e9e2660c40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9e25d8ba0 0x55e9e25dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9e25ccc10 0x55e9e25e1000 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 --2- 172.21.15.67:0/2157703511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9e25cc2f0 0x55e9e25e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.941+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 >> 172.21.15.67:0/2157703511 conn(0x55e9e25ca120 msgr2=0x55e9e25d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.942+0000 7f78f3400640 0 client.16386 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.942+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 shutdown_connections 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.942+0000 7f7916e2c740 1 -- 172.21.15.67:0/2157703511 wait complete. 2024-09-10T23:19:46.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.942+0000 7f7916e2c740 20 client.16386 trim_cache size 0 max 16384 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 62, 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "f8f54173": { 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 62, 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:19:46.945 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:19:46.946 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 msgr2=0x7f78f40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 0x7f78f40b79c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f790c076f90 tx=0x7f78f800b390 comp rx=0 tx=0).stop 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 msgr2=0x7f790c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7900009230 tx=0x7f7900004270 comp rx=0 tx=0).stop 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 shutdown_connections 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78f40b5500 0x7f78f40b79c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f790c108fa0 0x7f790c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f790c108620 0x7f790c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 --2- 172.21.15.67:0/335527628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f790c102c10 0x7f790c0754a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:19:46.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 >> 172.21.15.67:0/335527628 conn(0x7f790c0fe660 msgr2=0x7f790c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:19:46.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 shutdown_connections 2024-09-10T23:19:46.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:19:46.943+0000 7f7912e00640 1 -- 172.21.15.67:0/335527628 wait complete. 2024-09-10T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:47 smithi092 ceph-mon[34223]: pgmap v1042: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:19:48.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:47 smithi067 ceph-mon[30939]: pgmap v1042: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:19:48.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:47 smithi145 ceph-mon[34397]: pgmap v1042: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:19:48.424 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f8f54173', quiesce took 10.0 seconds. Will release after: 20.0 2024-09-10T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:49 smithi092 ceph-mon[34223]: pgmap v1043: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:50.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:49 smithi067 ceph-mon[30939]: pgmap v1043: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:50.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:49 smithi145 ceph-mon[34397]: pgmap v1043: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 11 op/s 2024-09-10T23:19:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:51 smithi092 ceph-mon[34223]: pgmap v1044: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:19:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:51 smithi067 ceph-mon[30939]: pgmap v1044: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:19:52.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:51 smithi145 ceph-mon[34397]: pgmap v1044: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:19:54.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:53 smithi092 ceph-mon[34223]: pgmap v1045: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:19:54.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:53 smithi067 ceph-mon[30939]: pgmap v1045: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:19:54.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:53 smithi145 ceph-mon[34397]: pgmap v1045: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-10T23:19:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:55 smithi092 ceph-mon[34223]: pgmap v1046: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 11 op/s 2024-09-10T23:19:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:55 smithi067 ceph-mon[30939]: pgmap v1046: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 11 op/s 2024-09-10T23:19:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:55 smithi145 ceph-mon[34397]: pgmap v1046: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 11 op/s 2024-09-10T23:19:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:57 smithi092 ceph-mon[34223]: pgmap v1047: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-10T23:19:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:57 smithi067 ceph-mon[30939]: pgmap v1047: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-10T23:19:58.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:57 smithi145 ceph-mon[34397]: pgmap v1047: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-10T23:20:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:19:59 smithi092 ceph-mon[34223]: pgmap v1048: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-09-10T23:20:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:19:59 smithi067 ceph-mon[30939]: pgmap v1048: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-09-10T23:20:00.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:19:59 smithi145 ceph-mon[34397]: pgmap v1048: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-09-10T23:20:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:00 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T23:20:01.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:00 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T23:20:01.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:00 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T23:20:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:01 smithi092 ceph-mon[34223]: pgmap v1049: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 10 op/s 2024-09-10T23:20:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:01 smithi067 ceph-mon[30939]: pgmap v1049: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 10 op/s 2024-09-10T23:20:02.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:01 smithi145 ceph-mon[34397]: pgmap v1049: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 10 op/s 2024-09-10T23:20:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:03 smithi092 ceph-mon[34223]: pgmap v1050: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 21 op/s 2024-09-10T23:20:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:03 smithi067 ceph-mon[30939]: pgmap v1050: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 21 op/s 2024-09-10T23:20:04.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:03 smithi145 ceph-mon[34397]: pgmap v1050: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 21 op/s 2024-09-10T23:20:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:05 smithi092 ceph-mon[34223]: pgmap v1051: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 660 KiB/s wr, 23 op/s 2024-09-10T23:20:06.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:05 smithi067 ceph-mon[30939]: pgmap v1051: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 660 KiB/s wr, 23 op/s 2024-09-10T23:20:06.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:05 smithi145 ceph-mon[34397]: pgmap v1051: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 660 KiB/s wr, 23 op/s 2024-09-10T23:20:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:08 smithi092 ceph-mon[34223]: pgmap v1052: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 660 KiB/s wr, 25 op/s 2024-09-10T23:20:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:08 smithi067 ceph-mon[30939]: pgmap v1052: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 660 KiB/s wr, 25 op/s 2024-09-10T23:20:08.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:08 smithi145 ceph-mon[34397]: pgmap v1052: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 660 KiB/s wr, 25 op/s 2024-09-10T23:20:08.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:20:08.822 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:09.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:09 smithi067 ceph-mon[30939]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T23:20:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:09 smithi092 ceph-mon[34223]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T23:20:09.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:09 smithi145 ceph-mon[34397]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T23:20:09.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 -- 172.21.15.67:0/4040451138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010b330 msgr2=0x7fd4b0111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:09.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4040451138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010b330 0x7fd4b0111b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009ca0 tx=0x7fd4a802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 -- 172.21.15.67:0/4040451138 shutdown_connections 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4040451138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010b330 0x7fd4b0111b30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4040451138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b010a9b0 0x7fd4b010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4040451138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b0104fa0 0x7fd4b0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 -- 172.21.15.67:0/4040451138 >> 172.21.15.67:0/4040451138 conn(0x7fd4b0100a50 msgr2=0x7fd4b0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 -- 172.21.15.67:0/4040451138 shutdown_connections 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.576+0000 7fd4b6400640 1 -- 172.21.15.67:0/4040451138 wait complete. 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.577+0000 7fd4b6400640 1 Processor -- start 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 -- start start 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 0x7fd4b019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:09.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b010b330 0x7fd4b01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0114e60 con 0x7fd4b010a9b0 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0114ce0 con 0x7fd4b010b330 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4b6400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0114fe0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4af400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 0x7fd4b019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36208/0 (socket says 172.21.15.67:36208) 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.578+0000 7fd4aea00640 1 -- 172.21.15.67:0/4073639225 learned_addr learned my addr 172.21.15.67:0/4073639225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4afe00640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b010b330 0x7fd4b01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 -- 172.21.15.67:0/4073639225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b010b330 msgr2=0x7fd4b01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b010b330 0x7fd4b01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 -- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 msgr2=0x7fd4b019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 -- 172.21.15.67:0/4073639225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01a1400 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4aea00640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.579+0000 7fd4af400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 0x7fd4b019b350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000e9f0 tx=0x7fd4a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.580+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000ccc0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.580+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4a00108c0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.580+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00187e0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.580+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b01a16f0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.580+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b01a7db0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.581+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b0105380 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.583+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd4a0010430 con 0x7fd4b0104fa0 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.585+0000 7fd4a7e00640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 0x7fd4940b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:09.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.585+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7fd4a00e66c0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.585+0000 7fd4aea00640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 0x7fd4940b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:09.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.586+0000 7fd4aea00640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 0x7fd4940b7990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01096e0 tx=0x7fd49c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:09.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.590+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd4a00ac750 con 0x7fd4b0104fa0 2024-09-10T23:20:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.830+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4b002e5c0 con 0x7fd4b0104fa0 2024-09-10T23:20:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.831+0000 7fd4a7e00640 1 -- 172.21.15.67:0/4073639225 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fd4a00b1180 con 0x7fd4b0104fa0 2024-09-10T23:20:09.832 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:20:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:20:09.832 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:20:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 msgr2=0x7fd4940b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 0x7fd4940b7990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01096e0 tx=0x7fd49c005f90 comp rx=0 tx=0).stop 2024-09-10T23:20:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 msgr2=0x7fd4b019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 0x7fd4b019b350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000e9f0 tx=0x7fd4a000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 shutdown_connections 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd4940b54d0 0x7fd4940b7990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b010b330 0x7fd4b01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b010a9b0 0x7fd4b019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 --2- 172.21.15.67:0/4073639225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4b0104fa0 0x7fd4b019b350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.834+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 >> 172.21.15.67:0/4073639225 conn(0x7fd4b0100a50 msgr2=0x7fd4b0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.835+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 shutdown_connections 2024-09-10T23:20:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:09.835+0000 7fd4b6400640 1 -- 172.21.15.67:0/4073639225 wait complete. 2024-09-10T23:20:10.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:10 smithi067 ceph-mon[30939]: pgmap v1054: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-10T23:20:10.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:10 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:20:10.259 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:10 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4073639225' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:10.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:10 smithi092 ceph-mon[34223]: pgmap v1054: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-10T23:20:10.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:10 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:20:10.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:10 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4073639225' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:10 smithi145 ceph-mon[34397]: pgmap v1054: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-10T23:20:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:10 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:20:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:10 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4073639225' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:10.714 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id f8f54173 --release --await' 2024-09-10T23:20:10.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id f8f54173 --release --await 2024-09-10T23:20:11.094 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:20:11.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:20:11.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:20:11.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:20:11.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:20:11.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:20:11.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:20:11.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:20:11.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 -- 172.21.15.67:0/1427576557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 msgr2=0x7f4334075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 --2- 172.21.15.67:0/1427576557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f4334075920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f432402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 -- 172.21.15.67:0/1427576557 shutdown_connections 2024-09-10T23:20:11.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 --2- 172.21.15.67:0/1427576557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 0x7f4334111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 --2- 172.21.15.67:0/1427576557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 --2- 172.21.15.67:0/1427576557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f4334075920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 -- 172.21.15.67:0/1427576557 >> 172.21.15.67:0/1427576557 conn(0x7f43340fe6d0 msgr2=0x7f4334100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 -- 172.21.15.67:0/1427576557 shutdown_connections 2024-09-10T23:20:11.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.775+0000 7f4338c00640 1 -- 172.21.15.67:0/1427576557 wait complete. 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.776+0000 7f4338c00640 1 Processor -- start 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 -- start start 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f433419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 0x7f43341a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4334114cd0 con 0x7f4334075f40 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4334114b50 con 0x7f433410f220 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f4338c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4334114e50 con 0x7f4334075540 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f432aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f432aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36212/0 (socket says 172.21.15.67:36212) 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.777+0000 7f432aa00640 1 -- 172.21.15.67:0/1667369204 learned_addr learned my addr 172.21.15.67:0/1667369204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f4333e00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 0x7f43341a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f4333400640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f433419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 -- 172.21.15.67:0/1667369204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 msgr2=0x7f433419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f433419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 -- 172.21.15.67:0/1667369204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 msgr2=0x7f43341a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 0x7f43341a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 -- 172.21.15.67:0/1667369204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43341a12b0 con 0x7f4334075f40 2024-09-10T23:20:11.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.778+0000 7f432aa00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f431c00ea60 tx=0x7f431c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.779+0000 7f4330c00640 1 -- 172.21.15.67:0/1667369204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c00cde0 con 0x7f4334075f40 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.779+0000 7f4330c00640 1 -- 172.21.15.67:0/1667369204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f431c0035f0 con 0x7f4334075f40 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.779+0000 7f4330c00640 1 -- 172.21.15.67:0/1667369204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c010690 con 0x7f4334075f40 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.779+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43341a15a0 con 0x7f4334075f40 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.779+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a7e30 con 0x7f4334075f40 2024-09-10T23:20:11.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.782+0000 7f4330c00640 1 -- 172.21.15.67:0/1667369204 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f431c004000 con 0x7f4334075f40 2024-09-10T23:20:11.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.784+0000 7f4330c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 0x7f43180b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.784+0000 7f4330c00640 1 -- 172.21.15.67:0/1667369204 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7f431c0e66c0 con 0x7f4334075f40 2024-09-10T23:20:11.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.784+0000 7f4333400640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 0x7f43180b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.785+0000 7f4333400640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 0x7f43180b7a40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4324006010 tx=0x7f4324005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.804+0000 7f433cc82740 1 Processor -- start 2024-09-10T23:20:11.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.804+0000 7f433cc82740 1 -- start start 2024-09-10T23:20:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6f1d0 0x556dfdc6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc7de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfdbe6290 con 0x556dfdc6faf0 2024-09-10T23:20:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556dfdbd56c0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f433cc82740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556dfdbd94d0 con 0x556dfdc7ba80 2024-09-10T23:20:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f432aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f432aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36228/0 (socket says 172.21.15.67:36228) 2024-09-10T23:20:11.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f432aa00640 1 -- 172.21.15.67:0/2513810346 learned_addr learned my addr 172.21.15.67:0/2513810346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f4333e00640 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc7de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.805+0000 7f4333400640 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6f1d0 0x556dfdc6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 -- 172.21.15.67:0/2513810346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 msgr2=0x556dfdc7de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc7de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 -- 172.21.15.67:0/2513810346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6f1d0 msgr2=0x556dfdc6f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6f1d0 0x556dfdc6f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 -- 172.21.15.67:0/2513810346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfdb62460 con 0x556dfdc6faf0 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f432aa00640 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ac3a0 tx=0x7f431c0ac470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c585200137df1e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f4323e00640 1 -- 172.21.15.67:0/2513810346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0ac1b0 con 0x556dfdc6faf0 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f4323e00640 1 -- 172.21.15.67:0/2513810346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f431c0acda0 con 0x556dfdc6faf0 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.806+0000 7f4323e00640 1 -- 172.21.15.67:0/2513810346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0a8910 con 0x556dfdc6faf0 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 -- 172.21.15.67:0/2513810346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 msgr2=0x556dfdc7b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ac3a0 tx=0x7f431c0ac470 comp rx=0 tx=0).stop 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 -- 172.21.15.67:0/2513810346 shutdown_connections 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc7de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6faf0 0x556dfdc7b540 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 --2- 172.21.15.67:0/2513810346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6f1d0 0x556dfdc6f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 -- 172.21.15.67:0/2513810346 >> 172.21.15.67:0/2513810346 conn(0x556dfdc6d000 msgr2=0x556dfdc6e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 -- 172.21.15.67:0/2513810346 shutdown_connections 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.807+0000 7f433cc82740 1 -- 172.21.15.67:0/2513810346 wait complete. 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.808+0000 7f433cc82740 1 -- start start 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.808+0000 7f433cc82740 1 Processor -- start 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc78f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfda36bb0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556dfdb706c0 con 0x556dfdc7ba80 2024-09-10T23:20:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556dfdc7e670 con 0x556dfdc6faf0 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f433cc82740 10 client.0 _ll_register_callbacks cb 0x556dfdb8b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36234/0 (socket says 172.21.15.67:36234) 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f4333400640 1 -- 172.21.15.67:0/246701668 learned_addr learned my addr 172.21.15.67:0/246701668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f432aa00640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc78f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.809+0000 7f4333e00640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 -- 172.21.15.67:0/246701668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 msgr2=0x556dfdc78f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc78f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 -- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 msgr2=0x556dfdc794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc794d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 -- 172.21.15.67:0/246701668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfd933b00 con 0x556dfdc6f1d0 2024-09-10T23:20:11.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333e00640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:11.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4333400640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f4324009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c31a80c6f5a4dde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4320c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324035040 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7f432404a430 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.810+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f431c0acda0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f432404a670 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 10 client.16404 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556dfd9aa960 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f431c0acda0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4324003a80 con 0x556dfdc6f1d0 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 10 client.16404 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 10 client.16404 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 20 client.16404 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdc8b5a0 0x556dfdc8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 4 client.16404 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.811+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dfdbe81e0 con 0x556dfdc8b5a0 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.812+0000 7f432aa00640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdc8b5a0 0x556dfdc8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.813+0000 7f432aa00640 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdc8b5a0 0x556dfdc8d9e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a9d38618570641 server_cookie=9a348e74781934e5 in_seq=0 out_seq=0 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.813+0000 7f4320c00640 10 client.16404 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.813+0000 7f4320c00640 1 -- 172.21.15.67:0/246701668 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556dfdbe81e0 con 0x556dfdc8b5a0 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.813+0000 7f4320c00640 10 client.16404 handle_command_reply: tid=0 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.813+0000 7f433cc82740 1 client.16404 shutdown 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 msgr2=0x556dfdc78a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f4324009260 comp rx=0 tx=0).stop 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 shutdown_connections 2024-09-10T23:20:11.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdc8b5a0 0x556dfdc8d9e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc78f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 --2- 172.21.15.67:0/246701668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc78a50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 >> 172.21.15.67:0/246701668 conn(0x556dfdc6d000 msgr2=0x556dfdc72ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f4320c00640 0 client.16404 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 shutdown_connections 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 1 -- 172.21.15.67:0/246701668 wait complete. 2024-09-10T23:20:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.814+0000 7f433cc82740 20 client.16404 trim_cache size 0 max 16384 2024-09-10T23:20:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.832+0000 7f433cc82740 1 Processor -- start 2024-09-10T23:20:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.832+0000 7f433cc82740 1 -- start start 2024-09-10T23:20:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc7a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc75c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b4960 con 0x556dfdc6f1d0 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b2230 con 0x556dfdc7ba80 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f433cc82740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b00a0 con 0x556dfdc6faf0 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36240/0 (socket says 172.21.15.67:36240) 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f4333400640 1 -- 172.21.15.67:0/3295838124 learned_addr learned my addr 172.21.15.67:0/3295838124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f4333e00640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc75c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.833+0000 7f432aa00640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc7a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 -- 172.21.15.67:0/3295838124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 msgr2=0x556dfdc7a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc7a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 -- 172.21.15.67:0/3295838124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 msgr2=0x556dfdc75c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc75c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 -- 172.21.15.67:0/3295838124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfdc76f50 con 0x556dfdc6f1d0 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333e00640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc75c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4333400640 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f432402a9e0 tx=0x7f4324000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ed95a1d4419d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4323400640 1 -- 172.21.15.67:0/3295838124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324035070 con 0x556dfdc6f1d0 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.834+0000 7f4323400640 1 -- 172.21.15.67:0/3295838124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f432404a430 con 0x556dfdc6f1d0 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f4323400640 1 -- 172.21.15.67:0/3295838124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240035f0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 -- 172.21.15.67:0/3295838124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 msgr2=0x556dfdc73f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f432402a9e0 tx=0x7f4324000bc0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 -- 172.21.15.67:0/3295838124 shutdown_connections 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc7ba80 0x556dfdc75c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc6faf0 0x556dfdc7a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 --2- 172.21.15.67:0/3295838124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc73f00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.835+0000 7f433cc82740 1 -- 172.21.15.67:0/3295838124 >> 172.21.15.67:0/3295838124 conn(0x556dfdc6d000 msgr2=0x556dfdc77af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.836+0000 7f433cc82740 1 -- 172.21.15.67:0/3295838124 shutdown_connections 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.836+0000 7f433cc82740 1 -- 172.21.15.67:0/3295838124 wait complete. 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.836+0000 7f433cc82740 1 -- start start 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.836+0000 7f433cc82740 1 Processor -- start 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 0x556dfdc80410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc80950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b06f0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b1b60 con 0x556dfdc6faf0 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b0dc0 con 0x556dfdc7ba80 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f433cc82740 10 client.0 _ll_register_callbacks cb 0x7f432c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f4333400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36254/0 (socket says 172.21.15.67:36254) 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f4333400640 1 -- 172.21.15.67:0/1117614940 learned_addr learned my addr 172.21.15.67:0/1117614940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f432aa00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 0x556dfdc80410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f4333400640 1 -- 172.21.15.67:0/1117614940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 msgr2=0x556dfdc80950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.837+0000 7f4333e00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc80950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333400640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc80950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333400640 1 -- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 msgr2=0x556dfdc80410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333400640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 0x556dfdc80410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333400640 1 -- 172.21.15.67:0/1117614940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfdc80f90 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f432aa00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 0x556dfdc80410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333e00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc80950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4333400640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4324061040 tx=0x7f432404a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94595fe79deb9321 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.838+0000 7f4320c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324035040 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7f43240035f0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432404a430 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43240037f0 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f433cc82740 10 client.16410 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556dfdcf9370 con 0x556dfdc6f1d0 2024-09-10T23:20:11.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.839+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f432404a430 con 0x556dfdc6f1d0 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4324070030 con 0x556dfdc6f1d0 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 10 client.16410 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 10 client.16410 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 20 client.16410 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdcf9870 0x556dfdcfbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 4 client.16410 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f8f54173", "release": true, "await": true}] 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f8f54173", "release": true, "await": true}) -- 0x556dfdbe3c90 con 0x556dfdcf9870 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.840+0000 7f432aa00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdcf9870 0x556dfdcfbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.841+0000 7f432aa00640 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdcf9870 0x556dfdcfbc90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e356490cde60806 server_cookie=129215f21f7582a7 in_seq=0 out_seq=0 2024-09-10T23:20:11.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.841+0000 7f4320c00640 10 client.16410 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.875+0000 7f4320c00640 1 -- 172.21.15.67:0/1117614940 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556dfdbe3c90 con 0x556dfdcf9870 2024-09-10T23:20:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.875+0000 7f4320c00640 10 client.16410 handle_command_reply: tid=0 2024-09-10T23:20:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.875+0000 7f433cc82740 1 client.16410 shutdown 2024-09-10T23:20:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 msgr2=0x556dfdc83ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4324061040 tx=0x7f432404a810 comp rx=0 tx=0).stop 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 shutdown_connections 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x556dfdcf9870 0x556dfdcfbc90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556dfdc7ba80 0x556dfdc80950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556dfdc6faf0 0x556dfdc80410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 --2- 172.21.15.67:0/1117614940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfdc6f1d0 0x556dfdc83ee0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f4320c00640 0 client.16410 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 >> 172.21.15.67:0/1117614940 conn(0x556dfdc6d000 msgr2=0x556dfdc772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 shutdown_connections 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.876+0000 7f433cc82740 1 -- 172.21.15.67:0/1117614940 wait complete. 2024-09-10T23:20:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.877+0000 7f433cc82740 20 client.16410 trim_cache size 0 max 16384 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 64, 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "f8f54173": { 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 64, 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:20:11.878 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:11.879 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 msgr2=0x7f43180b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 0x7f43180b7a40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4324006010 tx=0x7f4324005e80 comp rx=0 tx=0).stop 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 msgr2=0x7f433419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f431c00ea60 tx=0x7f431c00ef20 comp rx=0 tx=0).stop 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 shutdown_connections 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43180b5580 0x7f43180b7a40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f433410f220 0x7f43341a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4334075f40 0x7f433419b740 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 --2- 172.21.15.67:0/1667369204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4334075540 0x7f433419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:11.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 >> 172.21.15.67:0/1667369204 conn(0x7f43340fe6d0 msgr2=0x7f43340ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:11.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 shutdown_connections 2024-09-10T23:20:11.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:11.878+0000 7f4338c00640 1 -- 172.21.15.67:0/1667369204 wait complete. 2024-09-10T23:20:12.156 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:12 smithi067 ceph-mon[30939]: pgmap v1055: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:20:12.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:12 smithi092 ceph-mon[34223]: pgmap v1055: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:20:12.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:12 smithi145 ceph-mon[34397]: pgmap v1055: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:20:13.653 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f8f54173', total seconds elapsed: 35.2 2024-09-10T23:20:13.653 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:20:13.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:20:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:14.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:14 smithi067 ceph-mon[30939]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:20:14.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:14 smithi092 ceph-mon[34223]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:20:14.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:14 smithi145 ceph-mon[34397]: pgmap v1056: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.903+0000 7fe6d5c00640 1 -- 172.21.15.67:0/1245579212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d0103b10 msgr2=0x7fe6d0103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.903+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/1245579212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d0103b10 0x7fe6d0103ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009900 tx=0x7fe6c002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 -- 172.21.15.67:0/1245579212 shutdown_connections 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/1245579212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d010cf70 0x7fe6d010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/1245579212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0104430 0x7fe6d010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/1245579212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d0103b10 0x7fe6d0103ef0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 -- 172.21.15.67:0/1245579212 >> 172.21.15.67:0/1245579212 conn(0x7fe6d00fd5b0 msgr2=0x7fe6d00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 -- 172.21.15.67:0/1245579212 shutdown_connections 2024-09-10T23:20:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.904+0000 7fe6d5c00640 1 -- 172.21.15.67:0/1245579212 wait complete. 2024-09-10T23:20:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.905+0000 7fe6d5c00640 1 Processor -- start 2024-09-10T23:20:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.905+0000 7fe6d5c00640 1 -- start start 2024-09-10T23:20:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0103b10 0x7fe6d019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d0104430 0x7fe6d019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0112a90 con 0x7fe6d010cf70 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0112910 con 0x7fe6d0103b10 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0112c10 con 0x7fe6d0104430 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36266/0 (socket says 172.21.15.67:36266) 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6d5200640 1 -- 172.21.15.67:0/3298276394 learned_addr learned my addr 172.21.15.67:0/3298276394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6cf400640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d0104430 0x7fe6d019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:14.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.906+0000 7fe6cfe00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0103b10 0x7fe6d019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 -- 172.21.15.67:0/3298276394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d0104430 msgr2=0x7fe6d019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d0104430 0x7fe6d019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 -- 172.21.15.67:0/3298276394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0103b10 msgr2=0x7fe6d019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0103b10 0x7fe6d019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 -- 172.21.15.67:0/3298276394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d01a11e0 con 0x7fe6d010cf70 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5200640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800eaa0 tx=0x7fe6c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cdb0 con 0x7fe6d010cf70 2024-09-10T23:20:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c8005020 con 0x7fe6d010cf70 2024-09-10T23:20:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.907+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d01a14d0 con 0x7fe6d010cf70 2024-09-10T23:20:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.908+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d01a7b90 con 0x7fe6d010cf70 2024-09-10T23:20:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.908+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80116d0 con 0x7fe6d010cf70 2024-09-10T23:20:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.909+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d0103ef0 con 0x7fe6d010cf70 2024-09-10T23:20:14.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.911+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe6c800f040 con 0x7fe6d010cf70 2024-09-10T23:20:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.913+0000 7fe6ccc00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 0x7fe6b40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.913+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7fe6c80e6590 con 0x7fe6d010cf70 2024-09-10T23:20:14.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.913+0000 7fe6cfe00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 0x7fe6b40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:14.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.914+0000 7fe6cfe00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 0x7fe6b40b79c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00057c0 tx=0x7fe6c00058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:14.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:14.915+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6c8016030 con 0x7fe6d010cf70 2024-09-10T23:20:15.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.206+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d01a7ea0 con 0x7fe6d010cf70 2024-09-10T23:20:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.207+0000 7fe6ccc00640 1 -- 172.21.15.67:0/3298276394 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fe6c80ac620 con 0x7fe6d010cf70 2024-09-10T23:20:15.208 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:20:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:20:15.208 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:20:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.209+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 msgr2=0x7fe6b40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.209+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 0x7fe6b40b79c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00057c0 tx=0x7fe6c00058d0 comp rx=0 tx=0).stop 2024-09-10T23:20:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.209+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 msgr2=0x7fe6d01a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:15.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.209+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800eaa0 tx=0x7fe6c800ef60 comp rx=0 tx=0).stop 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 shutdown_connections 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe6b40b5500 0x7fe6b40b79c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6d010cf70 0x7fe6d01a0a10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6d0104430 0x7fe6d019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 --2- 172.21.15.67:0/3298276394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d0103b10 0x7fe6d019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 >> 172.21.15.67:0/3298276394 conn(0x7fe6d00fd5b0 msgr2=0x7fe6d010f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 shutdown_connections 2024-09-10T23:20:15.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:15.210+0000 7fe6d5c00640 1 -- 172.21.15.67:0/3298276394 wait complete. 2024-09-10T23:20:16.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:16 smithi067 ceph-mon[30939]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:20:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:16 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3298276394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:16 smithi145 ceph-mon[34397]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:20:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:16 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3298276394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:16.407 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:20:16.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:20:16.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:16 smithi092 ceph-mon[34223]: pgmap v1057: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 820 KiB/s wr, 7 op/s 2024-09-10T23:20:16.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:16 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3298276394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 -- 172.21.15.67:0/304672454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf9410d100 msgr2=0x7faf9410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 --2- 172.21.15.67:0/304672454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf9410d100 0x7faf9410f560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009ca0 tx=0x7faf8c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 -- 172.21.15.67:0/304672454 shutdown_connections 2024-09-10T23:20:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 --2- 172.21.15.67:0/304672454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf9410d100 0x7faf9410f560 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 --2- 172.21.15.67:0/304672454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf941045c0 0x7faf9410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 --2- 172.21.15.67:0/304672454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf94104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 -- 172.21.15.67:0/304672454 >> 172.21.15.67:0/304672454 conn(0x7faf940fd700 msgr2=0x7faf940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 -- 172.21.15.67:0/304672454 shutdown_connections 2024-09-10T23:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.016+0000 7faf99600640 1 -- 172.21.15.67:0/304672454 wait complete. 2024-09-10T23:20:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.017+0000 7faf99600640 1 Processor -- start 2024-09-10T23:20:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.017+0000 7faf99600640 1 -- start start 2024-09-10T23:20:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf9419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9410d100 0x7faf941a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf94112c70 con 0x7faf941045c0 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf94112af0 con 0x7faf9410d100 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf99600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf94112df0 con 0x7faf94103ca0 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf9419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36276/0 (socket says 172.21.15.67:36276) 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf92000640 1 -- 172.21.15.67:0/656043013 learned_addr learned my addr 172.21.15.67:0/656043013 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf9419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:51702/0 (socket says 172.21.15.67:51702) 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.018+0000 7faf93400640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9410d100 0x7faf941a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 -- 172.21.15.67:0/656043013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 msgr2=0x7faf9419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf9419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 -- 172.21.15.67:0/656043013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9410d100 msgr2=0x7faf941a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9410d100 0x7faf941a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 -- 172.21.15.67:0/656043013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf941a1430 con 0x7faf941045c0 2024-09-10T23:20:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf92000640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9c0 tx=0x7faf8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.019+0000 7faf8b400640 1 -- 172.21.15.67:0/656043013 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8000ccc0 con 0x7faf941045c0 2024-09-10T23:20:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.020+0000 7faf8b400640 1 -- 172.21.15.67:0/656043013 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf800044b0 con 0x7faf941045c0 2024-09-10T23:20:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.020+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf941a1720 con 0x7faf941045c0 2024-09-10T23:20:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.020+0000 7faf8b400640 1 -- 172.21.15.67:0/656043013 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80003e50 con 0x7faf941045c0 2024-09-10T23:20:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.020+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf941a7fb0 con 0x7faf941045c0 2024-09-10T23:20:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.023+0000 7faf8b400640 1 -- 172.21.15.67:0/656043013 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faf80004020 con 0x7faf941045c0 2024-09-10T23:20:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.025+0000 7faf8b400640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 0x7faf780b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.025+0000 7faf8b400640 1 -- 172.21.15.67:0/656043013 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7faf800e6740 con 0x7faf941045c0 2024-09-10T23:20:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.025+0000 7faf92a00640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 0x7faf780b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.026+0000 7faf92a00640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 0x7faf780b7b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf84009bd0 tx=0x7faf84005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.045+0000 7faf9c03a740 1 Processor -- start 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.045+0000 7faf9c03a740 1 -- start start 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c37821ca0 con 0x558c378acaa0 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c378110d0 con 0x558c378ac180 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf9c03a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c37814ee0 con 0x558c378b8a30 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36290/0 (socket says 172.21.15.67:36290) 2024-09-10T23:20:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf92000640 1 -- 172.21.15.67:0/3153381595 learned_addr learned my addr 172.21.15.67:0/3153381595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:18.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf92a00640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.046+0000 7faf93400640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 -- 172.21.15.67:0/3153381595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 msgr2=0x558c378bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 -- 172.21.15.67:0/3153381595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 msgr2=0x558c378ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 -- 172.21.15.67:0/3153381595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c377c3400 con 0x558c378acaa0 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92a00640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf92000640 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faf800a8cf0 tx=0x7faf800a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fd5431fb53ad94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.047+0000 7faf7f400640 1 -- 172.21.15.67:0/3153381595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80093e60 con 0x558c378acaa0 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.048+0000 7faf7f400640 1 -- 172.21.15.67:0/3153381595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf800ac230 con 0x558c378acaa0 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.048+0000 7faf7f400640 1 -- 172.21.15.67:0/3153381595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800b5cb0 con 0x558c378acaa0 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.048+0000 7faf9c03a740 1 -- 172.21.15.67:0/3153381595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 msgr2=0x558c378b84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.048+0000 7faf9c03a740 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faf800a8cf0 tx=0x7faf800a86e0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 -- 172.21.15.67:0/3153381595 shutdown_connections 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b84f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 --2- 172.21.15.67:0/3153381595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 -- 172.21.15.67:0/3153381595 >> 172.21.15.67:0/3153381595 conn(0x558c378a9fb0 msgr2=0x558c378ab150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 -- 172.21.15.67:0/3153381595 shutdown_connections 2024-09-10T23:20:18.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 -- 172.21.15.67:0/3153381595 wait complete. 2024-09-10T23:20:18.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.049+0000 7faf9c03a740 1 -- start start 2024-09-10T23:20:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 Processor -- start 2024-09-10T23:20:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558c376727a0 con 0x558c378acaa0 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c377b6a10 con 0x558c378ac180 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c378bb6a0 con 0x558c378b8a30 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.050+0000 7faf9c03a740 10 client.0 _ll_register_callbacks cb 0x558c377c6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36298/0 (socket says 172.21.15.67:36298) 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 -- 172.21.15.67:0/2459210635 learned_addr learned my addr 172.21.15.67:0/2459210635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92a00640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf93400640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 -- 172.21.15.67:0/2459210635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 msgr2=0x558c378b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 -- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 msgr2=0x558c378b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 -- 172.21.15.67:0/2459210635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3757eb90 con 0x558c378acaa0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92a00640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf92000640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faf800e3a50 tx=0x7faf800a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c201e8d30ecb2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.051+0000 7faf73e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80097070 con 0x558c378acaa0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf800ac230 con 0x558c378acaa0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7faf800ab980 con 0x558c378acaa0 2024-09-10T23:20:18.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf80093e60 con 0x558c378acaa0 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf9c03a740 10 client.16428 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558c378c6f50 con 0x558c378acaa0 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.052+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf800ac230 con 0x558c378acaa0 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7faf800c9ce0 con 0x558c378acaa0 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 10 client.16428 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 10 client.16428 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 20 client.16428 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c378ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 4 client.16428 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c37823bf0 con 0x558c378c82a0 2024-09-10T23:20:18.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.053+0000 7faf92a00640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c378ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.054+0000 7faf92a00640 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c378ca690 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602c82df13d79c5b server_cookie=52f53ef0fa1ae05b in_seq=0 out_seq=0 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.054+0000 7faf73e00640 10 client.16428 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf73e00640 1 -- 172.21.15.67:0/2459210635 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c37823bf0 con 0x558c378c82a0 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf73e00640 10 client.16428 handle_command_reply: tid=0 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 client.16428 shutdown 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 msgr2=0x558c378b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faf800e3a50 tx=0x7faf800a8c40 comp rx=0 tx=0).stop 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 shutdown_connections 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c378ca690 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378b8a30 0x558c378b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378acaa0 0x558c378b56f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf73e00640 0 client.16428 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2459210635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 >> 172.21.15.67:0/2459210635 conn(0x558c378a9fb0 msgr2=0x558c378afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.055+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 shutdown_connections 2024-09-10T23:20:18.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.056+0000 7faf9c03a740 1 -- 172.21.15.67:0/2459210635 wait complete. 2024-09-10T23:20:18.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.056+0000 7faf9c03a740 20 client.16428 trim_cache size 0 max 16384 2024-09-10T23:20:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 Processor -- start 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 -- start start 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378acaa0 0x558c378aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b4b70 con 0x558c378b8a30 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b2440 con 0x558c378ac180 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.079+0000 7faf9c03a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b02b0 con 0x558c378acaa0 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36300/0 (socket says 172.21.15.67:36300) 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf93400640 1 -- 172.21.15.67:0/4113268709 learned_addr learned my addr 172.21.15.67:0/4113268709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf92a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf92000640 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378acaa0 0x558c378aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf92a00640 1 -- 172.21.15.67:0/4113268709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378acaa0 msgr2=0x558c378aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.080+0000 7faf92a00640 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378acaa0 0x558c378aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf92a00640 1 -- 172.21.15.67:0/4113268709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 msgr2=0x558c378b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf92a00640 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf92a00640 1 -- 172.21.15.67:0/4113268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c378b1020 con 0x558c378ac180 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf93400640 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf92a00640 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558c378bb760 tx=0x7faf84014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a922a42f9e1e3de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf7ea00640 1 -- 172.21.15.67:0/4113268709 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84035070 con 0x558c378ac180 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf7ea00640 1 -- 172.21.15.67:0/4113268709 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf84030b00 con 0x558c378ac180 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.081+0000 7faf7ea00640 1 -- 172.21.15.67:0/4113268709 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8401d9e0 con 0x558c378ac180 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 -- 172.21.15.67:0/4113268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 msgr2=0x558c378b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558c378bb760 tx=0x7faf84014000 comp rx=0 tx=0).stop 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 -- 172.21.15.67:0/4113268709 shutdown_connections 2024-09-10T23:20:18.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c378b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378acaa0 0x558c378aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 --2- 172.21.15.67:0/4113268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378ac180 0x558c378b4f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 -- 172.21.15.67:0/4113268709 >> 172.21.15.67:0/4113268709 conn(0x558c378a9fb0 msgr2=0x558c378b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 -- 172.21.15.67:0/4113268709 shutdown_connections 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.082+0000 7faf9c03a740 1 -- 172.21.15.67:0/4113268709 wait complete. 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.083+0000 7faf9c03a740 1 -- start start 2024-09-10T23:20:18.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.083+0000 7faf9c03a740 1 Processor -- start 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378ac180 0x558c378cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 0x558c378cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b0900 con 0x558c378b8a30 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b1d70 con 0x558c378acaa0 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf941b0fd0 con 0x558c378ac180 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf9c03a740 10 client.0 _ll_register_callbacks cb 0x7faf800d2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36306/0 (socket says 172.21.15.67:36306) 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf93400640 1 -- 172.21.15.67:0/2397176048 learned_addr learned my addr 172.21.15.67:0/2397176048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf92a00640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378ac180 0x558c378cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.084+0000 7faf92000640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 0x558c378cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 -- 172.21.15.67:0/2397176048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378ac180 msgr2=0x558c378cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378ac180 0x558c378cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 -- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 msgr2=0x558c378cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 0x558c378cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 -- 172.21.15.67:0/2397176048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c378ab370 con 0x558c378b8a30 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf92000640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 0x558c378cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf93400640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faf8c031c80 tx=0x7faf8c031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a517c6a4e5798c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf73e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c003d80 con 0x558c378b8a30 2024-09-10T23:20:18.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9607+0+0 (secure 0 0 0) 0x7faf8c004040 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.085+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf84030b00 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf8c054050 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 10 client.16434 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558c37935860 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf84030b00 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7faf8c04f030 con 0x558c378b8a30 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 10 client.16434 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 10 client.16434 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 20 client.16434 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c37938110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 4 client.16434 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:20:18.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.086+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x558c3781f6a0 con 0x558c378c82a0 2024-09-10T23:20:18.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.087+0000 7faf92a00640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c37938110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:18.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.088+0000 7faf92a00640 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c37938110 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a29acd0a3906bf server_cookie=369199d647cff6c7 in_seq=0 out_seq=0 2024-09-10T23:20:18.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.088+0000 7faf73e00640 10 client.16434 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.209+0000 7faf73e00640 1 -- 172.21.15.67:0/2397176048 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558c3781f6a0 con 0x558c378c82a0 2024-09-10T23:20:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.209+0000 7faf73e00640 10 client.16434 handle_command_reply: tid=0 2024-09-10T23:20:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.209+0000 7faf9c03a740 1 client.16434 shutdown 2024-09-10T23:20:18.210 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:18 smithi067 ceph-mon[30939]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 MiB/s wr, 7 op/s 2024-09-10T23:20:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 msgr2=0x558c3792cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faf8c031c80 tx=0x7faf8c031cb0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 shutdown_connections 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558c378c82a0 0x558c37938110 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558c378b8a30 0x558c3792cdd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c378acaa0 0x558c378cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 --2- 172.21.15.67:0/2397176048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c378ac180 0x558c378cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 >> 172.21.15.67:0/2397176048 conn(0x558c378a9fb0 msgr2=0x558c378b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf73e00640 0 client.16434 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 shutdown_connections 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 1 -- 172.21.15.67:0/2397176048 wait complete. 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.210+0000 7faf9c03a740 20 client.16434 trim_cache size 0 max 16384 2024-09-10T23:20:18.212 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 66, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "5336fb5f": { 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 66, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:20:18.213 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:18.214 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 msgr2=0x7faf780b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 0x7faf780b7b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf84009bd0 tx=0x7faf84005f90 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 msgr2=0x7faf9419b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9c0 tx=0x7faf8000ee80 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 shutdown_connections 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7faf780b5670 0x7faf780b7b30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9410d100 0x7faf941a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf941045c0 0x7faf9419b8c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 --2- 172.21.15.67:0/656043013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf94103ca0 0x7faf9419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.211+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 >> 172.21.15.67:0/656043013 conn(0x7faf940fd700 msgr2=0x7faf940fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.212+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 shutdown_connections 2024-09-10T23:20:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:18.212+0000 7faf99600640 1 -- 172.21.15.67:0/656043013 wait complete. 2024-09-10T23:20:18.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:18 smithi145 ceph-mon[34397]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 MiB/s wr, 7 op/s 2024-09-10T23:20:18.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:18 smithi092 ceph-mon[34223]: pgmap v1058: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 MiB/s wr, 7 op/s 2024-09-10T23:20:19.130 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5336fb5f', quiesce took 5.5 seconds. Will release after: 24.5 2024-09-10T23:20:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:19 smithi092 ceph-mon[34223]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 MiB/s wr, 7 op/s 2024-09-10T23:20:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:19 smithi092 ceph-mon[34223]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T23:20:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:19 smithi067 ceph-mon[30939]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 MiB/s wr, 7 op/s 2024-09-10T23:20:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:19 smithi067 ceph-mon[30939]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T23:20:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:19 smithi145 ceph-mon[34397]: pgmap v1059: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 MiB/s wr, 7 op/s 2024-09-10T23:20:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:19 smithi145 ceph-mon[34397]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T23:20:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:21 smithi092 ceph-mon[34223]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.1 MiB/s wr, 10 op/s 2024-09-10T23:20:22.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:21 smithi067 ceph-mon[30939]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.1 MiB/s wr, 10 op/s 2024-09-10T23:20:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:21 smithi145 ceph-mon[34397]: pgmap v1061: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.1 MiB/s wr, 10 op/s 2024-09-10T23:20:24.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:23 smithi092 ceph-mon[34223]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-10T23:20:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:23 smithi067 ceph-mon[30939]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-10T23:20:24.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:23 smithi145 ceph-mon[34397]: pgmap v1062: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.5 MiB/s wr, 10 op/s 2024-09-10T23:20:26.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:25 smithi092 ceph-mon[34223]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-09-10T23:20:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:25 smithi067 ceph-mon[30939]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-09-10T23:20:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:25 smithi145 ceph-mon[34397]: pgmap v1063: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.7 MiB/s wr, 10 op/s 2024-09-10T23:20:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:27 smithi092 ceph-mon[34223]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 MiB/s wr, 10 op/s 2024-09-10T23:20:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:27 smithi067 ceph-mon[30939]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 MiB/s wr, 10 op/s 2024-09-10T23:20:28.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:27 smithi145 ceph-mon[34397]: pgmap v1064: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 MiB/s wr, 10 op/s 2024-09-10T23:20:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:29 smithi092 ceph-mon[34223]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:20:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:29 smithi067 ceph-mon[30939]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:20:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:29 smithi145 ceph-mon[34397]: pgmap v1065: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-09-10T23:20:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:31 smithi092 ceph-mon[34223]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-09-10T23:20:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:31 smithi067 ceph-mon[30939]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-09-10T23:20:32.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:31 smithi145 ceph-mon[34397]: pgmap v1066: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-09-10T23:20:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:33 smithi092 ceph-mon[34223]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-10T23:20:34.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:33 smithi067 ceph-mon[30939]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-10T23:20:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:33 smithi145 ceph-mon[34397]: pgmap v1067: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 731 KiB/s wr, 7 op/s 2024-09-10T23:20:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:35 smithi092 ceph-mon[34223]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 390 KiB/s wr, 6 op/s 2024-09-10T23:20:36.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:35 smithi067 ceph-mon[30939]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 390 KiB/s wr, 6 op/s 2024-09-10T23:20:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:35 smithi145 ceph-mon[34397]: pgmap v1068: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 390 KiB/s wr, 6 op/s 2024-09-10T23:20:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:37 smithi092 ceph-mon[34223]: pgmap v1069: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 8 op/s 2024-09-10T23:20:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:37 smithi067 ceph-mon[30939]: pgmap v1069: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 8 op/s 2024-09-10T23:20:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:37 smithi145 ceph-mon[34397]: pgmap v1069: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 389 KiB/s wr, 8 op/s 2024-09-10T23:20:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:39 smithi092 ceph-mon[34223]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 47 KiB/s wr, 5 op/s 2024-09-10T23:20:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:39 smithi067 ceph-mon[30939]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 47 KiB/s wr, 5 op/s 2024-09-10T23:20:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:39 smithi145 ceph-mon[34397]: pgmap v1070: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 47 KiB/s wr, 5 op/s 2024-09-10T23:20:42.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:41 smithi092 ceph-mon[34223]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-09-10T23:20:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:41 smithi067 ceph-mon[30939]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-09-10T23:20:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:41 smithi145 ceph-mon[34397]: pgmap v1071: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-09-10T23:20:43.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:20:44.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:43 smithi092 ceph-mon[34223]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:44.038 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:44.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:43 smithi067 ceph-mon[30939]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:44.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:43 smithi145 ceph-mon[34397]: pgmap v1072: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 -- 172.21.15.67:0/3940504497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075540 msgr2=0x7fc8bc075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/3940504497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075540 0x7fc8bc075920 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009900 tx=0x7fc8b002a400 comp rx=0 tx=0).stop 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 -- 172.21.15.67:0/3940504497 shutdown_connections 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/3940504497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc10f3e0 0x7fc8bc1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/3940504497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc075f40 0x7fc8bc10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/3940504497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075540 0x7fc8bc075920 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 -- 172.21.15.67:0/3940504497 >> 172.21.15.67:0/3940504497 conn(0x7fc8bc0fe830 msgr2=0x7fc8bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 -- 172.21.15.67:0/3940504497 shutdown_connections 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.676+0000 7fc8c3c00640 1 -- 172.21.15.67:0/3940504497 wait complete. 2024-09-10T23:20:44.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.677+0000 7fc8c3c00640 1 Processor -- start 2024-09-10T23:20:44.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 -- start start 2024-09-10T23:20:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc075540 0x7fc8bc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc10f3e0 0x7fc8bc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114e80 con 0x7fc8bc075f40 2024-09-10T23:20:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114d00 con 0x7fc8bc10f3e0 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c3c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc115000 con 0x7fc8bc075540 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47294/0 (socket says 172.21.15.67:47294) 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc075540 0x7fc8bc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8bbe00640 1 -- 172.21.15.67:0/2098042545 learned_addr learned my addr 172.21.15.67:0/2098042545 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.678+0000 7fc8c1c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc10f3e0 0x7fc8bc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 -- 172.21.15.67:0/2098042545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc075540 msgr2=0x7fc8bc19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc075540 0x7fc8bc19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 -- 172.21.15.67:0/2098042545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc10f3e0 msgr2=0x7fc8bc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc10f3e0 0x7fc8bc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 -- 172.21.15.67:0/2098042545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc1a1380 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8bbe00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c920 tx=0x7fc8a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8007c70 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8a8007e10 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc1a1670 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.679+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8018710 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.680+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1a7d30 con 0x7fc8bc075f40 2024-09-10T23:20:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.681+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc075920 con 0x7fc8bc075f40 2024-09-10T23:20:44.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.684+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc8a800f450 con 0x7fc8bc075f40 2024-09-10T23:20:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.685+0000 7fc8b9600640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 0x7fc8a40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.685+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9519+0+0 (secure 0 0 0) 0x7fc8a80e67b0 con 0x7fc8bc075f40 2024-09-10T23:20:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.685+0000 7fc8c1200640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 0x7fc8a40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.686+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc8a8014030 con 0x7fc8bc075f40 2024-09-10T23:20:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.686+0000 7fc8c1200640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 0x7fc8a40b7910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0006010 tx=0x7fc8b0005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.922+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8bc1a80c0 con 0x7fc8bc075f40 2024-09-10T23:20:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.923+0000 7fc8b9600640 1 -- 172.21.15.67:0/2098042545 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11903 (secure 0 0 0) 0x7fc8a80acf80 con 0x7fc8bc075f40 2024-09-10T23:20:44.925 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:20:44.925 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"btime":"2024-09-10T23:17:27:742538+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:17:27.243207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:20:44.925 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 msgr2=0x7fc8a40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 0x7fc8a40b7910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0006010 tx=0x7fc8b0005e80 comp rx=0 tx=0).stop 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 msgr2=0x7fc8bc19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c920 tx=0x7fc8a800cde0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 shutdown_connections 2024-09-10T23:20:44.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fc8a40b5450 0x7fc8a40b7910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8bc10f3e0 0x7fc8bc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8bc075f40 0x7fc8bc19b810 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 --2- 172.21.15.67:0/2098042545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8bc075540 0x7fc8bc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.925+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 >> 172.21.15.67:0/2098042545 conn(0x7fc8bc0fe830 msgr2=0x7fc8bc1115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.926+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 shutdown_connections 2024-09-10T23:20:44.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:44.926+0000 7fc8c3c00640 1 -- 172.21.15.67:0/2098042545 wait complete. 2024-09-10T23:20:45.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:45 smithi067 ceph-mon[30939]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:45.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:45 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2098042545' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:45.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 5336fb5f --release --await' 2024-09-10T23:20:45.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 5336fb5f --release --await 2024-09-10T23:20:46.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:45 smithi092 ceph-mon[34223]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:46.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:45 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2098042545' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:45 smithi145 ceph-mon[34397]: pgmap v1073: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:20:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:45 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2098042545' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:20:46.376 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 -- 172.21.15.67:0/578916466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010b330 msgr2=0x7fb0d0111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 --2- 172.21.15.67:0/578916466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010b330 0x7fb0d0111b30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009ca0 tx=0x7fb0cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 -- 172.21.15.67:0/578916466 shutdown_connections 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 --2- 172.21.15.67:0/578916466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010b330 0x7fb0d0111b30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 --2- 172.21.15.67:0/578916466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010a9b0 0x7fb0d010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 --2- 172.21.15.67:0/578916466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 0x7fb0d0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 -- 172.21.15.67:0/578916466 >> 172.21.15.67:0/578916466 conn(0x7fb0d0100a50 msgr2=0x7fb0d0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 -- 172.21.15.67:0/578916466 shutdown_connections 2024-09-10T23:20:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.316+0000 7fb0d8800640 1 -- 172.21.15.67:0/578916466 wait complete. 2024-09-10T23:20:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.317+0000 7fb0d8800640 1 Processor -- start 2024-09-10T23:20:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 -- start start 2024-09-10T23:20:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 0x7fb0d019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010b330 0x7fb0d01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0114e60 con 0x7fb0d010a9b0 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0114ce0 con 0x7fb0d010b330 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d8800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0114fe0 con 0x7fb0d0104fa0 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47300/0 (socket says 172.21.15.67:47300) 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d5400640 1 -- 172.21.15.67:0/186042021 learned_addr learned my addr 172.21.15.67:0/186042021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.318+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 0x7fb0d019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d6800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010b330 0x7fb0d01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 -- 172.21.15.67:0/186042021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 msgr2=0x7fb0d019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 0x7fb0d019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 -- 172.21.15.67:0/186042021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010b330 msgr2=0x7fb0d01a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010b330 0x7fb0d01a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 -- 172.21.15.67:0/186042021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d01a14c0 con 0x7fb0d010a9b0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.319+0000 7fb0d5400640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d950 tx=0x7fb0c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.320+0000 7fb0caa00640 1 -- 172.21.15.67:0/186042021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0017030 con 0x7fb0d010a9b0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.320+0000 7fb0caa00640 1 -- 172.21.15.67:0/186042021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0c0004020 con 0x7fb0d010a9b0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.320+0000 7fb0caa00640 1 -- 172.21.15.67:0/186042021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00055d0 con 0x7fb0d010a9b0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.320+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d01a17b0 con 0x7fb0d010a9b0 2024-09-10T23:20:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.320+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d01a7e70 con 0x7fb0d010a9b0 2024-09-10T23:20:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.323+0000 7fb0db285740 1 Processor -- start 2024-09-10T23:20:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.323+0000 7fb0db285740 1 -- start start 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.323+0000 7fb0db285740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.323+0000 7fb0db285740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e596360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.323+0000 7fb0db285740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0db285740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56557e501290 con 0x56557e5968a0 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0db285740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56557e4f06c0 con 0x56557e589ff0 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0db285740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56557e4f44d0 con 0x56557e58a910 2024-09-10T23:20:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0caa00640 1 -- 172.21.15.67:0/186042021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb0c0009a30 con 0x7fb0d010a9b0 2024-09-10T23:20:47.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47304/0 (socket says 172.21.15.67:47304) 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0d5400640 1 -- 172.21.15.67:0/2921437691 learned_addr learned my addr 172.21.15.67:0/2921437691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0d6800640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.324+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e596360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 -- 172.21.15.67:0/2921437691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 msgr2=0x56557e596360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e596360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 -- 172.21.15.67:0/2921437691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 msgr2=0x56557e58a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 -- 172.21.15.67:0/2921437691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56557e47d460 con 0x56557e5968a0 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d6800640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00e9c60 tx=0x7fb0c00ee740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19dfb498f43f2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0bf000640 1 -- 172.21.15.67:0/2921437691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00f0650 con 0x56557e5968a0 2024-09-10T23:20:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0bf000640 1 -- 172.21.15.67:0/2921437691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0c00f0c80 con 0x56557e5968a0 2024-09-10T23:20:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.325+0000 7fb0bf000640 1 -- 172.21.15.67:0/2921437691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00f8b10 con 0x56557e5968a0 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.326+0000 7fb0caa00640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 0x7fb0b80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.326+0000 7fb0db285740 1 -- 172.21.15.67:0/2921437691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 msgr2=0x56557e598c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.326+0000 7fb0db285740 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00e9c60 tx=0x7fb0c00ee740 comp rx=0 tx=0).stop 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.326+0000 7fb0caa00640 1 -- 172.21.15.67:0/186042021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9519+0+0 (secure 0 0 0) 0x7fb0c00e5c80 con 0x7fb0d010a9b0 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.326+0000 7fb0db285740 1 -- 172.21.15.67:0/2921437691 shutdown_connections 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e598c90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e596360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 --2- 172.21.15.67:0/2921437691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 -- 172.21.15.67:0/2921437691 >> 172.21.15.67:0/2921437691 conn(0x56557e587e20 msgr2=0x56557e588fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0d6800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 0x7fb0b80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 -- 172.21.15.67:0/2921437691 shutdown_connections 2024-09-10T23:20:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 -- 172.21.15.67:0/2921437691 wait complete. 2024-09-10T23:20:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 -- start start 2024-09-10T23:20:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.327+0000 7fb0db285740 1 Processor -- start 2024-09-10T23:20:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0d6800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 0x7fb0b80b7990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc006010 tx=0x7fb0cc009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e593870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e593db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56557e351bb0 con 0x56557e5968a0 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56557e48b6c0 con 0x56557e589ff0 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56557e599550 con 0x56557e58a910 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0db285740 10 client.0 _ll_register_callbacks cb 0x56557e4a6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47312/0 (socket says 172.21.15.67:47312) 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e593db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.328+0000 7fb0d6800640 1 -- 172.21.15.67:0/578871996 learned_addr learned my addr 172.21.15.67:0/578871996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e593870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 -- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 msgr2=0x56557e593870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e593870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 -- 172.21.15.67:0/578871996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 msgr2=0x56557e5942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5942f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 -- 172.21.15.67:0/578871996 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56557e24eb00 con 0x56557e58a910 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0d5400640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e593db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00ab9c0 tx=0x7fb0c00e9140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca08a2a4d1bbcbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.329+0000 7fb0b3400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00a8310 con 0x56557e58a910 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c00f0c80 con 0x56557e58a910 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9519+0+0 (secure 0 0 0) 0x7fb0c00a7f30 con 0x56557e58a910 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0c00ab430 con 0x56557e58a910 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0db285740 10 client.26122 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.330+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56557e2c5960 con 0x56557e58a910 2024-09-10T23:20:47.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0c00f0c80 con 0x56557e58a910 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb0c00ed040 con 0x56557e58a910 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 10 client.26122 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 10 client.26122 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 20 client.26122 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e5a62c0 0x56557e5a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 4 client.26122 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56557e5031e0 con 0x56557e5a62c0 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.331+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e5a62c0 0x56557e5a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.332+0000 7fb0b3400640 10 client.26122 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.332+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e5a62c0 0x56557e5a86b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294e392ef7e0425 server_cookie=5ef6089cd9883c63 in_seq=0 out_seq=0 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.333+0000 7fb0b3400640 1 -- 172.21.15.67:0/578871996 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56557e5031e0 con 0x56557e5a62c0 2024-09-10T23:20:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.333+0000 7fb0b3400640 10 client.26122 handle_command_reply: tid=0 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.333+0000 7fb0db285740 1 client.26122 shutdown 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 msgr2=0x56557e593db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e593db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00ab9c0 tx=0x7fb0c00e9140 comp rx=0 tx=0).stop 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 shutdown_connections 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e5a62c0 0x56557e5a86b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e58a910 0x56557e593db0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 --2- 172.21.15.67:0/578871996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e593870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 >> 172.21.15.67:0/578871996 conn(0x56557e587e20 msgr2=0x56557e58d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0b3400640 0 client.26122 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 shutdown_connections 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 1 -- 172.21.15.67:0/578871996 wait complete. 2024-09-10T23:20:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.334+0000 7fb0db285740 20 client.26122 trim_cache size 0 max 16384 2024-09-10T23:20:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.368+0000 7fb0db285740 1 Processor -- start 2024-09-10T23:20:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.368+0000 7fb0db285740 1 -- start start 2024-09-10T23:20:47.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.368+0000 7fb0db285740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.368+0000 7fb0db285740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0db285740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e5968a0 0x56557e590a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0db285740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b4ae0 con 0x56557e58a910 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0db285740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b23b0 con 0x56557e589ff0 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0db285740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b0220 con 0x56557e5968a0 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47322/0 (socket says 172.21.15.67:47322) 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 -- 172.21.15.67:0/2773811573 learned_addr learned my addr 172.21.15.67:0/2773811573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d6800640 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e5968a0 0x56557e590a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 -- 172.21.15.67:0/2773811573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e5968a0 msgr2=0x56557e590a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e5968a0 0x56557e590a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 -- 172.21.15.67:0/2773811573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 msgr2=0x56557e58ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.369+0000 7fb0d5400640 1 -- 172.21.15.67:0/2773811573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56557e5888a0 con 0x56557e58a910 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.370+0000 7fb0d5400640 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00a8f30 tx=0x7fb0c00ee740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d6ed8169adbd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.370+0000 7fb0be600640 1 -- 172.21.15.67:0/2773811573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00ab610 con 0x56557e58a910 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.370+0000 7fb0be600640 1 -- 172.21.15.67:0/2773811573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0c00e9500 con 0x56557e58a910 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.370+0000 7fb0be600640 1 -- 172.21.15.67:0/2773811573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00aba90 con 0x56557e58a910 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 -- 172.21.15.67:0/2773811573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 msgr2=0x56557e586c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00a8f30 tx=0x7fb0c00ee740 comp rx=0 tx=0).stop 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 -- 172.21.15.67:0/2773811573 shutdown_connections 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e5968a0 0x56557e590a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e58a910 0x56557e586c00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 --2- 172.21.15.67:0/2773811573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e589ff0 0x56557e58ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 -- 172.21.15.67:0/2773811573 >> 172.21.15.67:0/2773811573 conn(0x56557e587e20 msgr2=0x56557e5956c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 -- 172.21.15.67:0/2773811573 shutdown_connections 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.371+0000 7fb0db285740 1 -- 172.21.15.67:0/2773811573 wait complete. 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 -- start start 2024-09-10T23:20:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 Processor -- start 2024-09-10T23:20:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e589ff0 0x56557e5ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 0x56557e5a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b0870 con 0x56557e5968a0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b1ce0 con 0x56557e58a910 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01b0f40 con 0x56557e589ff0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.372+0000 7fb0db285740 10 client.0 _ll_register_callbacks cb 0x7fb0cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47338/0 (socket says 172.21.15.67:47338) 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 -- 172.21.15.67:0/28601023 learned_addr learned my addr 172.21.15.67:0/28601023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e589ff0 0x56557e5ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d5400640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 0x56557e5a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 -- 172.21.15.67:0/28601023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e589ff0 msgr2=0x56557e5ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e589ff0 0x56557e5ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 -- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 msgr2=0x56557e5a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 0x56557e5a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d6800640 1 -- 172.21.15.67:0/28601023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56557e5aa990 con 0x56557e5968a0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.373+0000 7fb0d5400640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 0x56557e5a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0d6800640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc035000 tx=0x7fb0cc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273fb99274624768 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0b3400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc003c40 con 0x56557e5968a0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9519+0+0 (secure 0 0 0) 0x7fb0cc003de0 con 0x56557e5968a0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c00e9500 con 0x56557e5968a0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.374+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0cc004050 con 0x56557e5968a0 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 10 client.16458 fetch_fsmap learned FSMap version 35 2024-09-10T23:20:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56557e613d60 con 0x56557e5968a0 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0c00e9500 con 0x56557e5968a0 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb0cc030b10 con 0x56557e5968a0 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 10 client.16458 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 10 client.16458 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 20 client.16458 populate_metadata read hostname 'smithi067' 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e614590 0x56557e6169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 4 client.16458 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5336fb5f", "release": true, "await": true}] 2024-09-10T23:20:47.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.375+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5336fb5f", "release": true, "await": true}) -- 0x56557e4fec90 con 0x56557e614590 2024-09-10T23:20:47.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.376+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e614590 0x56557e6169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:20:47.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.376+0000 7fb0d5e00640 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e614590 0x56557e6169b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b097cfacbb2ff5a server_cookie=5dc1ed37d673a0f5 in_seq=0 out_seq=0 2024-09-10T23:20:47.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.376+0000 7fb0b3400640 10 client.16458 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0b3400640 1 -- 172.21.15.67:0/28601023 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56557e4fec90 con 0x56557e614590 2024-09-10T23:20:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0b3400640 10 client.16458 handle_command_reply: tid=0 2024-09-10T23:20:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0db285740 1 client.16458 shutdown 2024-09-10T23:20:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 msgr2=0x56557e5aa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc035000 tx=0x7fb0cc041000 comp rx=0 tx=0).stop 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 shutdown_connections 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.402+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56557e614590 0x56557e6169b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56557e5968a0 0x56557e5aa320 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56557e58a910 0x56557e5a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 --2- 172.21.15.67:0/28601023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56557e589ff0 0x56557e5ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 >> 172.21.15.67:0/28601023 conn(0x56557e587e20 msgr2=0x56557e5a1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0b3400640 0 client.16458 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:20:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 shutdown_connections 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 1 -- 172.21.15.67:0/28601023 wait complete. 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.403+0000 7fb0db285740 20 client.16458 trim_cache size 0 max 16384 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:20:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 68, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "5336fb5f": { 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 68, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:20:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:20:47.408 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 msgr2=0x7fb0b80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 0x7fb0b80b7990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc006010 tx=0x7fb0cc009360 comp rx=0 tx=0).stop 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 msgr2=0x7fb0d019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d950 tx=0x7fb0c000de10 comp rx=0 tx=0).stop 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 shutdown_connections 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb0b80b54d0 0x7fb0b80b7990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d010b330 0x7fb0d01a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d010a9b0 0x7fb0d019b950 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 --2- 172.21.15.67:0/186042021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0104fa0 0x7fb0d019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:20:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.404+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 >> 172.21.15.67:0/186042021 conn(0x7fb0d0100a50 msgr2=0x7fb0d01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:20:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.405+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 shutdown_connections 2024-09-10T23:20:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:20:47.405+0000 7fb0d8800640 1 -- 172.21.15.67:0/186042021 wait complete. 2024-09-10T23:20:47.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:47 smithi067 ceph-mon[30939]: pgmap v1074: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:20:48.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:47 smithi092 ceph-mon[34223]: pgmap v1074: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:20:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:47 smithi145 ceph-mon[34397]: pgmap v1074: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T23:20:48.452 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5336fb5f', total seconds elapsed: 34.8 2024-09-10T23:20:50.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:49 smithi092 ceph-mon[34223]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:20:50.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:49 smithi067 ceph-mon[30939]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:20:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:49 smithi145 ceph-mon[34397]: pgmap v1075: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-09-10T23:20:52.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:51 smithi092 ceph-mon[34223]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 MiB/s wr, 7 op/s 2024-09-10T23:20:52.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:51 smithi067 ceph-mon[30939]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 MiB/s wr, 7 op/s 2024-09-10T23:20:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:51 smithi145 ceph-mon[34397]: pgmap v1076: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 MiB/s wr, 7 op/s 2024-09-10T23:20:54.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:53 smithi092 ceph-mon[34223]: pgmap v1077: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-09-10T23:20:54.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:53 smithi067 ceph-mon[30939]: pgmap v1077: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-09-10T23:20:54.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:53 smithi145 ceph-mon[34397]: pgmap v1077: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-09-10T23:20:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:55 smithi092 ceph-mon[34223]: pgmap v1078: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-10T23:20:56.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:55 smithi067 ceph-mon[30939]: pgmap v1078: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-10T23:20:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:55 smithi145 ceph-mon[34397]: pgmap v1078: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.6 MiB/s wr, 7 op/s 2024-09-10T23:20:58.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:57 smithi092 ceph-mon[34223]: pgmap v1079: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 10 op/s 2024-09-10T23:20:58.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:57 smithi067 ceph-mon[30939]: pgmap v1079: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 10 op/s 2024-09-10T23:20:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:57 smithi145 ceph-mon[34397]: pgmap v1079: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 10 op/s 2024-09-10T23:21:00.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:20:59 smithi092 ceph-mon[34223]: pgmap v1080: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9.6 MiB/s wr, 8 op/s 2024-09-10T23:21:00.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:20:59 smithi067 ceph-mon[30939]: pgmap v1080: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9.6 MiB/s wr, 8 op/s 2024-09-10T23:21:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:20:59 smithi145 ceph-mon[34397]: pgmap v1080: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9.6 MiB/s wr, 8 op/s 2024-09-10T23:21:02.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:01 smithi092 ceph-mon[34223]: pgmap v1081: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:02.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:01 smithi067 ceph-mon[30939]: pgmap v1081: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:01 smithi145 ceph-mon[34397]: pgmap v1081: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:03 smithi092 ceph-mon[34223]: pgmap v1082: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:21:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:03 smithi092 ceph-mon[34223]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T23:21:04.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:03 smithi067 ceph-mon[30939]: pgmap v1082: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:21:04.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:03 smithi067 ceph-mon[30939]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T23:21:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:03 smithi145 ceph-mon[34397]: pgmap v1082: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 26 op/s 2024-09-10T23:21:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:03 smithi145 ceph-mon[34397]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T23:21:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:05 smithi092 ceph-mon[34223]: pgmap v1084: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:05 smithi067 ceph-mon[30939]: pgmap v1084: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:06.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:05 smithi145 ceph-mon[34397]: pgmap v1084: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:08.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:07 smithi092 ceph-mon[34223]: pgmap v1085: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:08.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:07 smithi067 ceph-mon[30939]: pgmap v1085: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:08.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:07 smithi145 ceph-mon[34397]: pgmap v1085: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:10.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:09 smithi092 ceph-mon[34223]: pgmap v1086: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:10.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:09 smithi067 ceph-mon[30939]: pgmap v1086: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:09 smithi145 ceph-mon[34397]: pgmap v1086: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:11.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:10 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:21:11.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:10 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:21:11.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:10 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:21:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:11 smithi092 ceph-mon[34223]: pgmap v1087: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:21:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:21:12.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:11 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:21:12.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:11 smithi067 ceph-mon[30939]: pgmap v1087: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:21:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:21:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:11 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:21:12.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:11 smithi145 ceph-mon[34397]: pgmap v1087: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 33 op/s 2024-09-10T23:21:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:21:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:21:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:11 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:21:14.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:13 smithi092 ceph-mon[34223]: pgmap v1088: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T23:21:14.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:13 smithi092 ceph-mon[34223]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T23:21:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:13 smithi067 ceph-mon[30939]: pgmap v1088: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T23:21:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:13 smithi067 ceph-mon[30939]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T23:21:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:13 smithi145 ceph-mon[34397]: pgmap v1088: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T23:21:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:13 smithi145 ceph-mon[34397]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T23:21:16.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:15 smithi092 ceph-mon[34223]: pgmap v1090: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:16.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:15 smithi067 ceph-mon[30939]: pgmap v1090: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:16.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:15 smithi145 ceph-mon[34397]: pgmap v1090: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:18.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:17 smithi067 ceph-mon[30939]: pgmap v1091: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:21:18.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:17 smithi145 ceph-mon[34397]: pgmap v1091: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:21:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:17 smithi092 ceph-mon[34223]: pgmap v1091: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-10T23:21:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:19 smithi067 ceph-mon[30939]: pgmap v1092: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:20.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:19 smithi145 ceph-mon[34397]: pgmap v1092: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:19 smithi092 ceph-mon[34223]: pgmap v1092: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:21 smithi067 ceph-mon[30939]: pgmap v1093: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:22.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:21 smithi145 ceph-mon[34397]: pgmap v1093: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:21 smithi092 ceph-mon[34223]: pgmap v1093: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 13 op/s 2024-09-10T23:21:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:23 smithi067 ceph-mon[30939]: pgmap v1094: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:24.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:23 smithi145 ceph-mon[34397]: pgmap v1094: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:24.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:23 smithi092 ceph-mon[34223]: pgmap v1094: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:26.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:25 smithi067 ceph-mon[30939]: pgmap v1095: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:21:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:25 smithi145 ceph-mon[34397]: pgmap v1095: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:21:26.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:25 smithi092 ceph-mon[34223]: pgmap v1095: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-10T23:21:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:27 smithi067 ceph-mon[30939]: pgmap v1096: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:21:28.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:27 smithi092 ceph-mon[34223]: pgmap v1096: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:21:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:27 smithi145 ceph-mon[34397]: pgmap v1096: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 15 op/s 2024-09-10T23:21:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:29 smithi067 ceph-mon[30939]: pgmap v1097: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:21:30.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:29 smithi145 ceph-mon[34397]: pgmap v1097: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:21:30.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:29 smithi092 ceph-mon[34223]: pgmap v1097: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 11 op/s 2024-09-10T23:21:32.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:31 smithi067 ceph-mon[30939]: pgmap v1098: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:32.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:31 smithi145 ceph-mon[34397]: pgmap v1098: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:32.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:31 smithi092 ceph-mon[34223]: pgmap v1098: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:34.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:33 smithi145 ceph-mon[34397]: pgmap v1099: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:21:34.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:33 smithi092 ceph-mon[34223]: pgmap v1099: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:21:34.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:33 smithi067 ceph-mon[30939]: pgmap v1099: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-09-10T23:21:36.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:35 smithi145 ceph-mon[34397]: pgmap v1100: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:21:36.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:35 smithi092 ceph-mon[34223]: pgmap v1100: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:21:36.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:35 smithi067 ceph-mon[30939]: pgmap v1100: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 13 op/s 2024-09-10T23:21:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:37 smithi145 ceph-mon[34397]: pgmap v1101: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:21:38.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:37 smithi092 ceph-mon[34223]: pgmap v1101: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:21:38.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:37 smithi067 ceph-mon[30939]: pgmap v1101: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-09-10T23:21:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:39 smithi145 ceph-mon[34397]: pgmap v1102: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:21:40.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:39 smithi092 ceph-mon[34223]: pgmap v1102: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:21:40.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:39 smithi067 ceph-mon[30939]: pgmap v1102: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 12 op/s 2024-09-10T23:21:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:41 smithi145 ceph-mon[34397]: pgmap v1103: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:41 smithi092 ceph-mon[34223]: pgmap v1103: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:42.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:41 smithi067 ceph-mon[30939]: pgmap v1103: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:44.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:43 smithi145 ceph-mon[34397]: pgmap v1104: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:43 smithi092 ceph-mon[34223]: pgmap v1104: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:44.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:43 smithi067 ceph-mon[30939]: pgmap v1104: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 13 op/s 2024-09-10T23:21:46.099 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:45 smithi067 ceph-mon[30939]: pgmap v1105: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:45 smithi145 ceph-mon[34397]: pgmap v1105: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:46.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:45 smithi092 ceph-mon[34223]: pgmap v1105: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 15 op/s 2024-09-10T23:21:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:47 smithi145 ceph-mon[34397]: pgmap v1106: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:21:48.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:47 smithi067 ceph-mon[30939]: pgmap v1106: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:21:48.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:47 smithi092 ceph-mon[34223]: pgmap v1106: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-09-10T23:21:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:49 smithi145 ceph-mon[34397]: pgmap v1107: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:21:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:49 smithi092 ceph-mon[34223]: pgmap v1107: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:21:50.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:49 smithi067 ceph-mon[30939]: pgmap v1107: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-09-10T23:21:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:51 smithi145 ceph-mon[34397]: pgmap v1108: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:51 smithi092 ceph-mon[34223]: pgmap v1108: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:52.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:51 smithi067 ceph-mon[30939]: pgmap v1108: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-09-10T23:21:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:53 smithi145 ceph-mon[34397]: pgmap v1109: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:53 smithi092 ceph-mon[34223]: pgmap v1109: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:54.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:53 smithi067 ceph-mon[30939]: pgmap v1109: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:55 smithi145 ceph-mon[34397]: pgmap v1110: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:21:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:55 smithi092 ceph-mon[34223]: pgmap v1110: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:21:56.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:55 smithi067 ceph-mon[30939]: pgmap v1110: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-09-10T23:21:57.434 INFO:tasks.workunit.client.0.smithi067.stdout:fs setup took 1248 secs 2024-09-10T23:21:58.114 INFO:tasks.workunit.client.0.smithi067.stdout:Syncing()...0 sec 2024-09-10T23:21:58.116 INFO:tasks.workunit.client.0.smithi067.stdout:Starting Actual Benchmark At: Tue Sep 10 23:21:58 2024 2024-09-10T23:21:58.116 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:21:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:57 smithi145 ceph-mon[34397]: pgmap v1111: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:57 smithi092 ceph-mon[34223]: pgmap v1111: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:21:58.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:57 smithi067 ceph-mon[30939]: pgmap v1111: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:22:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:21:59 smithi145 ceph-mon[34397]: pgmap v1112: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:22:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:21:59 smithi092 ceph-mon[34223]: pgmap v1112: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:22:00.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:21:59 smithi067 ceph-mon[30939]: pgmap v1112: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-10T23:22:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:01 smithi145 ceph-mon[34397]: pgmap v1113: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 144 op/s 2024-09-10T23:22:02.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:01 smithi092 ceph-mon[34223]: pgmap v1113: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 144 op/s 2024-09-10T23:22:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:01 smithi067 ceph-mon[30939]: pgmap v1113: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 144 op/s 2024-09-10T23:22:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:03 smithi145 ceph-mon[34397]: pgmap v1114: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 262 op/s 2024-09-10T23:22:04.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:03 smithi092 ceph-mon[34223]: pgmap v1114: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 262 op/s 2024-09-10T23:22:04.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:03 smithi067 ceph-mon[30939]: pgmap v1114: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 262 op/s 2024-09-10T23:22:06.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:05 smithi145 ceph-mon[34397]: pgmap v1115: 129 pgs: 129 active+clean; 13 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.28k op/s 2024-09-10T23:22:06.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:05 smithi092 ceph-mon[34223]: pgmap v1115: 129 pgs: 129 active+clean; 13 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.28k op/s 2024-09-10T23:22:06.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:05 smithi067 ceph-mon[30939]: pgmap v1115: 129 pgs: 129 active+clean; 13 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.28k op/s 2024-09-10T23:22:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:07 smithi145 ceph-mon[34397]: pgmap v1116: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.46k op/s 2024-09-10T23:22:08.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:07 smithi092 ceph-mon[34223]: pgmap v1116: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.46k op/s 2024-09-10T23:22:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:07 smithi067 ceph-mon[30939]: pgmap v1116: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 1.46k op/s 2024-09-10T23:22:09.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:08 smithi092 ceph-mon[34223]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T23:22:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:08 smithi067 ceph-mon[30939]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T23:22:09.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:08 smithi145 ceph-mon[34397]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T23:22:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:09 smithi092 ceph-mon[34223]: pgmap v1118: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 2.33k op/s 2024-09-10T23:22:10.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:09 smithi067 ceph-mon[30939]: pgmap v1118: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 2.33k op/s 2024-09-10T23:22:10.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:09 smithi145 ceph-mon[34397]: pgmap v1118: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 2.33k op/s 2024-09-10T23:22:12.181 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:11 smithi145 ceph-mon[34397]: pgmap v1119: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T23:22:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:11 smithi092 ceph-mon[34223]: pgmap v1119: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T23:22:12.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:11 smithi067 ceph-mon[30939]: pgmap v1119: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T23:22:13.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:12 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:13.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:12 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:22:13.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:12 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:13.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:12 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:22:13.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:12 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:13.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:12 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:22:14.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:13 smithi092 ceph-mon[34223]: pgmap v1120: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:13 smithi067 ceph-mon[30939]: pgmap v1120: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:14.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:13 smithi145 ceph-mon[34397]: pgmap v1120: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:22:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:22:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:14 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:22:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:22:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:22:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:14 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:22:15.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:22:15.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:22:15.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:14 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:22:16.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:15 smithi092 ceph-mon[34223]: pgmap v1121: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T23:22:16.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:15 smithi067 ceph-mon[30939]: pgmap v1121: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T23:22:16.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:15 smithi145 ceph-mon[34397]: pgmap v1121: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T23:22:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:17 smithi092 ceph-mon[34223]: pgmap v1122: 129 pgs: 129 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:17 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:17 smithi067 ceph-mon[30939]: pgmap v1122: 129 pgs: 129 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:17 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:18.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:17 smithi145 ceph-mon[34397]: pgmap v1122: 129 pgs: 129 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.32k op/s 2024-09-10T23:22:18.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:17 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:22:20.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:19 smithi092 ceph-mon[34223]: pgmap v1123: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T23:22:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:19 smithi092 ceph-mon[34223]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T23:22:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:19 smithi067 ceph-mon[30939]: pgmap v1123: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T23:22:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:19 smithi067 ceph-mon[30939]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T23:22:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:19 smithi145 ceph-mon[34397]: pgmap v1123: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T23:22:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:19 smithi145 ceph-mon[34397]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T23:22:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:21 smithi092 ceph-mon[34223]: pgmap v1125: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:21 smithi067 ceph-mon[30939]: pgmap v1125: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:21 smithi145 ceph-mon[34397]: pgmap v1125: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:24.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:23 smithi092 ceph-mon[34223]: pgmap v1126: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:23 smithi067 ceph-mon[30939]: pgmap v1126: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:24.142 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:23 smithi145 ceph-mon[34397]: pgmap v1126: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T23:22:26.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:25 smithi092 ceph-mon[34223]: pgmap v1127: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-10T23:22:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:25 smithi067 ceph-mon[30939]: pgmap v1127: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-10T23:22:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:25 smithi145 ceph-mon[34397]: pgmap v1127: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-10T23:22:28.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:27 smithi092 ceph-mon[34223]: pgmap v1128: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:28.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:27 smithi067 ceph-mon[30939]: pgmap v1128: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:28.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:27 smithi145 ceph-mon[34397]: pgmap v1128: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:30.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:29 smithi092 ceph-mon[34223]: pgmap v1129: 129 pgs: 129 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-10T23:22:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:29 smithi067 ceph-mon[30939]: pgmap v1129: 129 pgs: 129 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-10T23:22:30.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:29 smithi145 ceph-mon[34397]: pgmap v1129: 129 pgs: 129 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-10T23:22:32.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:31 smithi092 ceph-mon[34223]: pgmap v1130: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:32.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:31 smithi067 ceph-mon[30939]: pgmap v1130: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:32.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:31 smithi145 ceph-mon[34397]: pgmap v1130: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:33 smithi092 ceph-mon[34223]: pgmap v1131: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:22:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:33 smithi067 ceph-mon[30939]: pgmap v1131: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:22:34.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:33 smithi145 ceph-mon[34397]: pgmap v1131: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:22:36.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:35 smithi092 ceph-mon[34223]: pgmap v1132: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T23:22:36.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:35 smithi067 ceph-mon[30939]: pgmap v1132: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T23:22:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:35 smithi145 ceph-mon[34397]: pgmap v1132: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T23:22:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:37 smithi092 ceph-mon[34223]: pgmap v1133: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T23:22:38.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:37 smithi067 ceph-mon[30939]: pgmap v1133: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T23:22:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:37 smithi145 ceph-mon[34397]: pgmap v1133: 129 pgs: 129 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T23:22:39.882 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:22:39.882 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-10T23:22:39.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:22:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:39 smithi067 ceph-mon[30939]: pgmap v1134: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T23:22:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:39 smithi145 ceph-mon[34397]: pgmap v1134: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T23:22:40.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:39 smithi092 ceph-mon[34223]: pgmap v1134: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T23:22:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:41 smithi067 ceph-mon[30939]: pgmap v1135: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:22:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:41 smithi145 ceph-mon[34397]: pgmap v1135: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:22:42.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:41 smithi092 ceph-mon[34223]: pgmap v1135: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:22:44.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:43 smithi067 ceph-mon[30939]: pgmap v1136: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 1.87k op/s 2024-09-10T23:22:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:43 smithi145 ceph-mon[34397]: pgmap v1136: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 1.87k op/s 2024-09-10T23:22:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:43 smithi092 ceph-mon[34223]: pgmap v1136: 129 pgs: 129 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 1.87k op/s 2024-09-10T23:22:46.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:45 smithi067 ceph-mon[30939]: pgmap v1137: 129 pgs: 129 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:22:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:45 smithi145 ceph-mon[34397]: pgmap v1137: 129 pgs: 129 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:22:46.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:45 smithi092 ceph-mon[34223]: pgmap v1137: 129 pgs: 129 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:22:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:47 smithi145 ceph-mon[34397]: pgmap v1138: 129 pgs: 129 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:22:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:47 smithi092 ceph-mon[34223]: pgmap v1138: 129 pgs: 129 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:22:48.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:47 smithi067 ceph-mon[30939]: pgmap v1138: 129 pgs: 129 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:22:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:49 smithi145 ceph-mon[34397]: pgmap v1139: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:22:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:49 smithi092 ceph-mon[34223]: pgmap v1139: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:22:50.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:49 smithi067 ceph-mon[30939]: pgmap v1139: 129 pgs: 129 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:22:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:51 smithi145 ceph-mon[34397]: pgmap v1140: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:52.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:51 smithi092 ceph-mon[34223]: pgmap v1140: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:52.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:51 smithi067 ceph-mon[30939]: pgmap v1140: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T23:22:54.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:53 smithi145 ceph-mon[34397]: pgmap v1141: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:22:54.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:53 smithi092 ceph-mon[34223]: pgmap v1141: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:22:54.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:53 smithi067 ceph-mon[30939]: pgmap v1141: 129 pgs: 129 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:22:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:55 smithi145 ceph-mon[34397]: pgmap v1142: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:55 smithi092 ceph-mon[34223]: pgmap v1142: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:56.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:55 smithi067 ceph-mon[30939]: pgmap v1142: 129 pgs: 129 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2024-09-10T23:22:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:57 smithi145 ceph-mon[34397]: pgmap v1143: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:22:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:57 smithi092 ceph-mon[34223]: pgmap v1143: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:22:58.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:57 smithi067 ceph-mon[30939]: pgmap v1143: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-09-10T23:23:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:22:59 smithi145 ceph-mon[34397]: pgmap v1144: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-10T23:23:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:22:59 smithi092 ceph-mon[34223]: pgmap v1144: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-10T23:23:00.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:22:59 smithi067 ceph-mon[30939]: pgmap v1144: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-10T23:23:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:01 smithi145 ceph-mon[34397]: pgmap v1145: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-10T23:23:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:01 smithi092 ceph-mon[34223]: pgmap v1145: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-10T23:23:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:01 smithi067 ceph-mon[30939]: pgmap v1145: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-09-10T23:23:03.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.587+0000 7f4947e00640 1 -- 172.21.15.67:0/1408827453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 msgr2=0x7f49481132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:03.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.587+0000 7f4947e00640 1 --2- 172.21.15.67:0/1408827453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f49481132c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4938009900 tx=0x7f493802a420 comp rx=0 tx=0).stop 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.588+0000 7f4947e00640 1 -- 172.21.15.67:0/1408827453 shutdown_connections 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.588+0000 7f4947e00640 1 --2- 172.21.15.67:0/1408827453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4948113800 0x7f4948115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.588+0000 7f4947e00640 1 --2- 172.21.15.67:0/1408827453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f49481132c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.588+0000 7f4947e00640 1 --2- 172.21.15.67:0/1408827453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49480776d0 0x7f4948077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.588+0000 7f4947e00640 1 -- 172.21.15.67:0/1408827453 >> 172.21.15.67:0/1408827453 conn(0x7f4948100840 msgr2=0x7f4948102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.589+0000 7f4947e00640 1 -- 172.21.15.67:0/1408827453 shutdown_connections 2024-09-10T23:23:03.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.589+0000 7f4947e00640 1 -- 172.21.15.67:0/1408827453 wait complete. 2024-09-10T23:23:03.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.592+0000 7f4947e00640 1 Processor -- start 2024-09-10T23:23:03.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.592+0000 7f4947e00640 1 -- start start 2024-09-10T23:23:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.596+0000 7f4947e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.596+0000 7f4947e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f494810d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:03.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.596+0000 7f4947e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4948113800 0x7f4948112750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:03.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4947e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49481051d0 con 0x7f4948113800 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4947e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4948105050 con 0x7f49480776d0 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4947e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4948105350 con 0x7f4948077ff0 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47550/0 (socket says 172.21.15.67:47550) 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 -- 172.21.15.67:0/1687208570 learned_addr learned my addr 172.21.15.67:0/1687208570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 -- 172.21.15.67:0/1687208570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 msgr2=0x7f494810d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946000640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f494810d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:03.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f494810d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 -- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4948113800 msgr2=0x7f4948112750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.597+0000 7f4946a00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4948113800 0x7f4948112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.598+0000 7f4946a00640 1 -- 172.21.15.67:0/1687208570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4948112f20 con 0x7f49480776d0 2024-09-10T23:23:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.598+0000 7f4946000640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f494810d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:23:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.598+0000 7f4946a00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f494000cce0 tx=0x7f494000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.601+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000f650 con 0x7f49480776d0 2024-09-10T23:23:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.601+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f494000fc80 con 0x7f49480776d0 2024-09-10T23:23:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.601+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940018ac0 con 0x7f49480776d0 2024-09-10T23:23:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.602+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4948113210 con 0x7f49480776d0 2024-09-10T23:23:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.603+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49481b4760 con 0x7f49480776d0 2024-09-10T23:23:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.605+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4948077ab0 con 0x7f49480776d0 2024-09-10T23:23:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.611+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4940020070 con 0x7f49480776d0 2024-09-10T23:23:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.613+0000 7f494ca00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 0x7f49280b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:03.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.614+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f49400e6aa0 con 0x7f49480776d0 2024-09-10T23:23:03.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.614+0000 7f4946000640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 0x7f49280b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:03.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.616+0000 7f4946000640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 0x7f49280b7a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4938009850 tx=0x7f4938000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:03.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:03.618+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f494001d060 con 0x7f49480776d0 2024-09-10T23:23:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.099+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f494810dfa0 con 0x7f49480776d0 2024-09-10T23:23:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.100+0000 7f494ca00640 1 -- 172.21.15.67:0/1687208570 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11906 (secure 0 0 0) 0x7f4940014030 con 0x7f49480776d0 2024-09-10T23:23:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:23:04.106 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-09-10T23:23:04.107 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"btime":"2024-09-10T23:22:16:959905+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:22:16.055341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:23:04.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 msgr2=0x7f49280b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:04.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 0x7f49280b7a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4938009850 tx=0x7f4938000bc0 comp rx=0 tx=0).stop 2024-09-10T23:23:04.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 msgr2=0x7f494810ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:04.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f494000cce0 tx=0x7f494000a620 comp rx=0 tx=0).stop 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 shutdown_connections 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f49280b55d0 0x7f49280b7a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4948113800 0x7f4948112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948077ff0 0x7f494810d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 --2- 172.21.15.67:0/1687208570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49480776d0 0x7f494810ce70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.105+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 >> 172.21.15.67:0/1687208570 conn(0x7f4948100840 msgr2=0x7f4948102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.107+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 shutdown_connections 2024-09-10T23:23:04.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:04.107+0000 7f4947e00640 1 -- 172.21.15.67:0/1687208570 wait complete. 2024-09-10T23:23:04.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:03 smithi067 ceph-mon[30939]: pgmap v1146: 129 pgs: 129 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:23:04.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:03 smithi067 ceph-mon[30939]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T23:23:04.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:03 smithi092 ceph-mon[34223]: pgmap v1146: 129 pgs: 129 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:23:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:03 smithi092 ceph-mon[34223]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T23:23:04.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:03 smithi145 ceph-mon[34397]: pgmap v1146: 129 pgs: 129 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:23:04.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:03 smithi145 ceph-mon[34397]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T23:23:05.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:04 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1687208570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:05.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:04 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1687208570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:05.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:04 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1687208570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:06.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:05 smithi092 ceph-mon[34223]: pgmap v1148: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T23:23:06.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:05 smithi067 ceph-mon[30939]: pgmap v1148: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T23:23:06.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:05 smithi145 ceph-mon[34397]: pgmap v1148: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T23:23:08.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:07 smithi092 ceph-mon[34223]: pgmap v1149: 129 pgs: 19 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 62 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:23:08.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:07 smithi067 ceph-mon[30939]: pgmap v1149: 129 pgs: 19 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 62 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:23:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:07 smithi145 ceph-mon[34397]: pgmap v1149: 129 pgs: 19 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 62 active+clean; 13 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:23:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:09 smithi092 ceph-mon[34223]: pgmap v1150: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T23:23:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:09 smithi067 ceph-mon[30939]: pgmap v1150: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T23:23:10.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:09 smithi145 ceph-mon[34397]: pgmap v1150: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T23:23:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:11 smithi092 ceph-mon[34223]: pgmap v1151: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:23:12.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:11 smithi067 ceph-mon[30939]: pgmap v1151: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:23:12.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:11 smithi145 ceph-mon[34397]: pgmap v1151: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:23:13.566 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-10T23:23:14.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-10T23:23:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:13 smithi092 ceph-mon[34223]: pgmap v1152: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:23:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:13 smithi067 ceph-mon[30939]: pgmap v1152: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:23:14.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:13 smithi145 ceph-mon[34397]: pgmap v1152: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:23:15.817 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:15 smithi092 ceph-mon[34223]: pgmap v1153: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T23:23:16.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:15 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:23:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:15 smithi067 ceph-mon[30939]: pgmap v1153: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T23:23:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:15 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:23:16.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:15 smithi145 ceph-mon[34397]: pgmap v1153: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T23:23:16.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:15 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:23:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:17 smithi092 ceph-mon[34223]: pgmap v1154: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-09-10T23:23:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:17 smithi067 ceph-mon[30939]: pgmap v1154: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-09-10T23:23:18.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:17 smithi145 ceph-mon[34397]: pgmap v1154: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-09-10T23:23:20.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:19 smithi092 ceph-mon[34223]: pgmap v1155: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:23:20.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:19 smithi067 ceph-mon[30939]: pgmap v1155: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:23:20.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:19 smithi145 ceph-mon[34397]: pgmap v1155: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:23:22.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:21 smithi092 ceph-mon[34223]: pgmap v1156: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.47k op/s 2024-09-10T23:23:22.301 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:21 smithi067 ceph-mon[30939]: pgmap v1156: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.47k op/s 2024-09-10T23:23:22.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:21 smithi145 ceph-mon[34397]: pgmap v1156: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.47k op/s 2024-09-10T23:23:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:23 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:23:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:23 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:23:23.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:23 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:23:23.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:23 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:23:23.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:23 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:23:23.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:23 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:23:23.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:23 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:23:23.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:23 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:23:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:23 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:23:24.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:24 smithi092 ceph-mon[34223]: pgmap v1157: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:23:24.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:24 smithi067 ceph-mon[30939]: pgmap v1157: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:23:24.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:24 smithi145 ceph-mon[34397]: pgmap v1157: 129 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:23:25.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.828+0000 7f93bee00640 1 -- 172.21.15.67:0/3723842659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 msgr2=0x7f93b8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.828+0000 7f93bee00640 1 --2- 172.21.15.67:0/3723842659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b8101630 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009be0 tx=0x7f93b002a6e0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 -- 172.21.15.67:0/3723842659 shutdown_connections 2024-09-10T23:23:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 --2- 172.21.15.67:0/3723842659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 --2- 172.21.15.67:0/3723842659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b8113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 --2- 172.21.15.67:0/3723842659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b8101630 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 -- 172.21.15.67:0/3723842659 >> 172.21.15.67:0/3723842659 conn(0x7f93b8078140 msgr2=0x7f93b80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 -- 172.21.15.67:0/3723842659 shutdown_connections 2024-09-10T23:23:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.836+0000 7f93bee00640 1 -- 172.21.15.67:0/3723842659 wait complete. 2024-09-10T23:23:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.838+0000 7f93bee00640 1 Processor -- start 2024-09-10T23:23:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.838+0000 7f93bee00640 1 -- start start 2024-09-10T23:23:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.839+0000 7f93bee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.839+0000 7f93bee00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.839+0000 7f93bee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.840+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.840+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59964/0 (socket says 172.21.15.67:59964) 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.842+0000 7f93b7400640 1 -- 172.21.15.67:0/1912309040 learned_addr learned my addr 172.21.15.67:0/1912309040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.842+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810ae70 con 0x7f93b8113860 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.842+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810acf0 con 0x7f93b8110f30 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.842+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810aff0 con 0x7f93b8101250 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.842+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93bce00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7400640 1 -- 172.21.15.67:0/1912309040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 msgr2=0x7f93b806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7400640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b806df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7400640 1 -- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 msgr2=0x7f93b8074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7400640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7400640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b8103f60 con 0x7f93b8110f30 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.843+0000 7f93bce00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.844+0000 7f93b7400640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93a801fd70 tx=0x7f93a801d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.845+0000 7f93b4c00640 1 -- 172.21.15.67:0/1912309040 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a801fe60 con 0x7f93b8110f30 2024-09-10T23:23:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.845+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b8104250 con 0x7f93b8110f30 2024-09-10T23:23:25.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.845+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81047e0 con 0x7f93b8110f30 2024-09-10T23:23:25.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.845+0000 7f93b4c00640 1 -- 172.21.15.67:0/1912309040 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a80035f0 con 0x7f93b8110f30 2024-09-10T23:23:25.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.845+0000 7f93b4c00640 1 -- 172.21.15.67:0/1912309040 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80216a0 con 0x7f93b8110f30 2024-09-10T23:23:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.853+0000 7f93b4c00640 1 -- 172.21.15.67:0/1912309040 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f93a8021840 con 0x7f93b8110f30 2024-09-10T23:23:25.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.856+0000 7f93b4c00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 0x7f939c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.856+0000 7f93b4c00640 1 -- 172.21.15.67:0/1912309040 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f93a80ba960 con 0x7f93b8110f30 2024-09-10T23:23:25.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.857+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 0x7f939c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.864+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 0x7f939c0b7990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006010 tx=0x7f93b0000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.865+0000 7f93c17e5740 1 Processor -- start 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 -- start start 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b117c40 0x558b1b1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b124900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558b1b08ba90 con 0x558b1b122510 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558b1b07aec0 con 0x558b1b1166e0 2024-09-10T23:23:25.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.866+0000 7f93c17e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b1b07ecd0 con 0x558b1b117c40 2024-09-10T23:23:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.867+0000 7f93b7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b117c40 0x558b1b1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59970/0 (socket says 172.21.15.67:59970) 2024-09-10T23:23:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 -- 172.21.15.67:0/1787154976 learned_addr learned my addr 172.21.15.67:0/1787154976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 -- 172.21.15.67:0/1787154976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b117c40 msgr2=0x558b1b1161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b117c40 0x558b1b1161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 -- 172.21.15.67:0/1787154976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 msgr2=0x558b1b124900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.868+0000 7f93b7400640 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b124900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.869+0000 7f93b7400640 1 -- 172.21.15.67:0/1787154976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1b0209d0 con 0x558b1b1166e0 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.869+0000 7f93b7400640 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f93a80bb590 tx=0x7f93a80c42f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8804128ae6acd43e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.870+0000 7f93ad600640 1 -- 172.21.15.67:0/1787154976 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80baee0 con 0x558b1b1166e0 2024-09-10T23:23:25.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.870+0000 7f93ad600640 1 -- 172.21.15.67:0/1787154976 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a802ee50 con 0x558b1b1166e0 2024-09-10T23:23:25.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.870+0000 7f93ad600640 1 -- 172.21.15.67:0/1787154976 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80bb080 con 0x558b1b1166e0 2024-09-10T23:23:25.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 -- 172.21.15.67:0/1787154976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 msgr2=0x558b1b116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f93a80bb590 tx=0x7f93a80c42f0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 -- 172.21.15.67:0/1787154976 shutdown_connections 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b124900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1166e0 0x558b1b116b00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1787154976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b117c40 0x558b1b1161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.874+0000 7f93c17e5740 1 -- 172.21.15.67:0/1787154976 >> 172.21.15.67:0/1787154976 conn(0x558b1b0e06f0 msgr2=0x558b1b113fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.875+0000 7f93c17e5740 1 -- 172.21.15.67:0/1787154976 shutdown_connections 2024-09-10T23:23:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.875+0000 7f93c17e5740 1 -- 172.21.15.67:0/1787154976 wait complete. 2024-09-10T23:23:25.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.877+0000 7f93c17e5740 1 -- start start 2024-09-10T23:23:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.877+0000 7f93c17e5740 1 Processor -- start 2024-09-10T23:23:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.879+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.879+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.880+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b122510 0x558b1b11ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.880+0000 7f93c17e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558b1aedc7b0 con 0x558b1b117c40 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.880+0000 7f93c17e5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558b1b01f650 con 0x558b1b122510 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.880+0000 7f93c17e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b1b125330 con 0x558b1b1166e0 2024-09-10T23:23:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.880+0000 7f93c17e5740 10 client.0 _ll_register_callbacks cb 0x558b1ad860a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:25.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.881+0000 7f93b7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.881+0000 7f93b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.881+0000 7f93bce00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b122510 0x558b1b11ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.882+0000 7f93b7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:36024/0 (socket says 172.21.15.67:36024) 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.882+0000 7f93b7e00640 1 -- 172.21.15.67:0/1449358768 learned_addr learned my addr 172.21.15.67:0/1449358768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.883+0000 7f93b7e00640 1 -- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b122510 msgr2=0x558b1b11ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.883+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b122510 0x558b1b11ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.883+0000 7f93b7e00640 1 -- 172.21.15.67:0/1449358768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 msgr2=0x558b1b11fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.883+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.883+0000 7f93b7e00640 1 -- 172.21.15.67:0/1449358768 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1ade8ba0 con 0x558b1b1166e0 2024-09-10T23:23:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.886+0000 7f93b7e00640 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034040 tx=0x7f93b0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13cf2f8cbf71e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.886+0000 7f93a2000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:23:25.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.887+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b004f070 con 0x558b1b1166e0 2024-09-10T23:23:25.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.887+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f93b0031440 con 0x558b1b1166e0 2024-09-10T23:23:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.887+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a802ee50 con 0x558b1b1166e0 2024-09-10T23:23:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.888+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f93b0059050 con 0x558b1b1166e0 2024-09-10T23:23:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.888+0000 7f93c17e5740 10 client.26143 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.888+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558b1b130f80 con 0x558b1b1166e0 2024-09-10T23:23:25.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a802ee50 con 0x558b1b1166e0 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f93b003c040 con 0x558b1b1166e0 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 10 client.26143 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 10 client.26143 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 20 client.26143 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1324e0 0x558b1b134910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 4 client.26143 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.889+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b1b08d9e0 con 0x558b1b1324e0 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.890+0000 7f93b7400640 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1324e0 0x558b1b134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.890+0000 7f93b7400640 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1324e0 0x558b1b134910 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96fe1e09af0dd91 server_cookie=e01309eb8398188e in_seq=0 out_seq=0 2024-09-10T23:23:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.890+0000 7f93a2000640 10 client.26143 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.891+0000 7f93a2000640 1 -- 172.21.15.67:0/1449358768 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b1b08d9e0 con 0x558b1b1324e0 2024-09-10T23:23:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.891+0000 7f93a2000640 10 client.26143 handle_command_reply: tid=0 2024-09-10T23:23:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.891+0000 7f93c17e5740 1 client.26143 shutdown 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 msgr2=0x558b1b11f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034040 tx=0x7f93b0009500 comp rx=0 tx=0).stop 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 shutdown_connections 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1324e0 0x558b1b134910 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b122510 0x558b1b11ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 --2- 172.21.15.67:0/1449358768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1166e0 0x558b1b11f510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 >> 172.21.15.67:0/1449358768 conn(0x558b1b0e06f0 msgr2=0x558b1b119b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.892+0000 7f93a2000640 0 client.26143 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.893+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 shutdown_connections 2024-09-10T23:23:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.893+0000 7f93c17e5740 1 -- 172.21.15.67:0/1449358768 wait complete. 2024-09-10T23:23:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.893+0000 7f93c17e5740 20 client.26143 trim_cache size 0 max 16384 2024-09-10T23:23:25.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.935+0000 7f93c17e5740 1 Processor -- start 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.935+0000 7f93c17e5740 1 -- start start 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1141c0 0x558b1b11cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9398012a90 con 0x558b1b117c40 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9398010300 con 0x558b1b1141c0 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93c17e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f939800e170 con 0x558b1b122510 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93b7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.936+0000 7f93b7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:36034/0 (socket says 172.21.15.67:36034) 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 -- 172.21.15.67:0/832630460 learned_addr learned my addr 172.21.15.67:0/832630460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93bce00640 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1141c0 0x558b1b11cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 -- 172.21.15.67:0/832630460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1141c0 msgr2=0x558b1b11cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1141c0 0x558b1b11cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 -- 172.21.15.67:0/832630460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 msgr2=0x558b1b11e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.937+0000 7f93b7400640 1 -- 172.21.15.67:0/832630460 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1b114ee0 con 0x558b1b122510 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.938+0000 7f93b7400640 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93a80be3f0 tx=0x7f93a80c1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c675f917bfe369 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.938+0000 7f93acc00640 1 -- 172.21.15.67:0/832630460 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80bb250 con 0x558b1b122510 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.938+0000 7f93acc00640 1 -- 172.21.15.67:0/832630460 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a80e5030 con 0x558b1b122510 2024-09-10T23:23:25.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.938+0000 7f93acc00640 1 -- 172.21.15.67:0/832630460 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80bb250 con 0x558b1b122510 2024-09-10T23:23:25.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.939+0000 7f93c17e5740 1 -- 172.21.15.67:0/832630460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 msgr2=0x558b1b11c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:25.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.939+0000 7f93c17e5740 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93a80be3f0 tx=0x7f93a80c1040 comp rx=0 tx=0).stop 2024-09-10T23:23:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 -- 172.21.15.67:0/832630460 shutdown_connections 2024-09-10T23:23:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b1141c0 0x558b1b11cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b122510 0x558b1b11c720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 --2- 172.21.15.67:0/832630460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b117c40 0x558b1b11e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 -- 172.21.15.67:0/832630460 >> 172.21.15.67:0/832630460 conn(0x558b1b0e06f0 msgr2=0x558b1b11dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 -- 172.21.15.67:0/832630460 shutdown_connections 2024-09-10T23:23:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.942+0000 7f93c17e5740 1 -- 172.21.15.67:0/832630460 wait complete. 2024-09-10T23:23:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.943+0000 7f93c17e5740 1 -- start start 2024-09-10T23:23:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.943+0000 7f93c17e5740 1 Processor -- start 2024-09-10T23:23:25.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.945+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1141c0 0x558b1b127d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.947+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.947+0000 7f93c17e5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b19f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.947+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.947+0000 7f93b7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59996/0 (socket says 172.21.15.67:59996) 2024-09-10T23:23:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93b7400640 1 -- 172.21.15.67:0/2522927230 learned_addr learned my addr 172.21.15.67:0/2522927230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f939800e7c0 con 0x558b1b122510 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f939800fc30 con 0x558b1b117c40 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f939800ee90 con 0x558b1b1141c0 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93c17e5740 10 client.0 _ll_register_callbacks cb 0x7f93a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.948+0000 7f93b7e00640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1141c0 0x558b1b127d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.949+0000 7f93b7400640 1 -- 172.21.15.67:0/2522927230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1141c0 msgr2=0x558b1b127d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.949+0000 7f93b7400640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1141c0 0x558b1b127d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.949+0000 7f93b7400640 1 -- 172.21.15.67:0/2522927230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 msgr2=0x558b1b19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.949+0000 7f93b7400640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.949+0000 7f93b7400640 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1b19fa20 con 0x558b1b117c40 2024-09-10T23:23:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.950+0000 7f93b7400640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f93a80ba840 tx=0x7f93a8005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1370a4b842da5c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.950+0000 7f93a2000640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:23:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.950+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80bb3f0 con 0x558b1b117c40 2024-09-10T23:23:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.951+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f93a80be6c0 con 0x558b1b117c40 2024-09-10T23:23:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.951+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a80e5030 con 0x558b1b117c40 2024-09-10T23:23:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.951+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f93a80bb5c0 con 0x558b1b117c40 2024-09-10T23:23:25.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.951+0000 7f93c17e5740 10 client.36179 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:25.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.951+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558b1b1a0d00 con 0x558b1b117c40 2024-09-10T23:23:25.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.952+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93a80e5030 con 0x558b1b117c40 2024-09-10T23:23:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f93a8021840 con 0x558b1b117c40 2024-09-10T23:23:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 10 client.36179 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 10 client.36179 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 20 client.36179 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1a0f30 0x558b1b1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 4 client.36179 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-10T23:23:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.955+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x558b1b089490 con 0x558b1b1a0f30 2024-09-10T23:23:25.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.956+0000 7f93b7e00640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1a0f30 0x558b1b1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:25.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.958+0000 7f93b7e00640 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1a0f30 0x558b1b1a3320 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6544d44f7e8bbd9a server_cookie=c848792cfaf86bf0 in_seq=0 out_seq=0 2024-09-10T23:23:25.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:25.958+0000 7f93a2000640 10 client.36179 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:26.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:26 smithi067 ceph-mon[30939]: pgmap v1158: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2024-09-10T23:23:26.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:26 smithi145 ceph-mon[34397]: pgmap v1158: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2024-09-10T23:23:26.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:26 smithi092 ceph-mon[34223]: pgmap v1158: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2024-09-10T23:23:28.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:28 smithi067 ceph-mon[30939]: pgmap v1159: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-09-10T23:23:28.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:28 smithi145 ceph-mon[34397]: pgmap v1159: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-09-10T23:23:28.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:28 smithi092 ceph-mon[34223]: pgmap v1159: 129 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-09-10T23:23:30.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:30 smithi067 ceph-mon[30939]: pgmap v1160: 129 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 98 active+clean; 14 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-10T23:23:30.323 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:30 smithi092 ceph-mon[34223]: pgmap v1160: 129 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 98 active+clean; 14 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-10T23:23:30.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:30 smithi145 ceph-mon[34397]: pgmap v1160: 129 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 98 active+clean; 14 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-10T23:23:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.061+0000 7f93a2000640 1 -- 172.21.15.67:0/2522927230 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558b1b089490 con 0x558b1b1a0f30 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.061+0000 7f93a2000640 10 client.36179 handle_command_reply: tid=0 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.061+0000 7f93c17e5740 1 client.36179 shutdown 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 msgr2=0x558b1b128260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f93a80ba840 tx=0x7f93a8005c80 comp rx=0 tx=0).stop 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 shutdown_connections 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x558b1b1a0f30 0x558b1b1a3320 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558b1b122510 0x558b1b19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558b1b117c40 0x558b1b128260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 --2- 172.21.15.67:0/2522927230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1b1141c0 0x558b1b127d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93a2000640 0 client.36179 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 >> 172.21.15.67:0/2522927230 conn(0x558b1b0e06f0 msgr2=0x558b1b11e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 shutdown_connections 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 1 -- 172.21.15.67:0/2522927230 wait complete. 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.062+0000 7f93c17e5740 20 client.36179 trim_cache size 0 max 16384 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 37, 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 70, 2024-09-10T23:23:31.065 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "bb620b90": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 70, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.8, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:31.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:23:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 msgr2=0x7f939c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 0x7f939c0b7990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006010 tx=0x7f93b0000f40 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 msgr2=0x7f93b806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93a801fd70 tx=0x7f93a801d6b0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 shutdown_connections 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f939c0b54d0 0x7f939c0b7990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93b8113860 0x7f93b8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b8110f30 0x7f93b806e440 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 --2- 172.21.15.67:0/1912309040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93b8101250 0x7f93b806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 >> 172.21.15.67:0/1912309040 conn(0x7f93b8078140 msgr2=0x7f93b8114100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 shutdown_connections 2024-09-10T23:23:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:31.064+0000 7f93bee00640 1 -- 172.21.15.67:0/1912309040 wait complete. 2024-09-10T23:23:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:32 smithi067 ceph-mon[30939]: pgmap v1161: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:23:32.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:32 smithi145 ceph-mon[34397]: pgmap v1161: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:23:32.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:32 smithi092 ceph-mon[34223]: pgmap v1161: 129 pgs: 9 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 107 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:23:34.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:34 smithi145 ceph-mon[34397]: pgmap v1162: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:23:34.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:34 smithi067 ceph-mon[30939]: pgmap v1162: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:23:34.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:34 smithi092 ceph-mon[34223]: pgmap v1162: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:23:34.539 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb620b90', quiesce took 54.7 seconds. Will release after: -35.85 2024-09-10T23:23:34.539 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:23:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 -- 172.21.15.67:0/1790625651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef410a800 msgr2=0x7fbef410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 --2- 172.21.15.67:0/1790625651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef410a800 0x7fbef410ac40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009900 tx=0x7fbee002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 -- 172.21.15.67:0/1790625651 shutdown_connections 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 --2- 172.21.15.67:0/1790625651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410b180 0x7fbef4111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 --2- 172.21.15.67:0/1790625651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef410a800 0x7fbef410ac40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 --2- 172.21.15.67:0/1790625651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef4104df0 0x7fbef41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 -- 172.21.15.67:0/1790625651 >> 172.21.15.67:0/1790625651 conn(0x7fbef4100840 msgr2=0x7fbef4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 -- 172.21.15.67:0/1790625651 shutdown_connections 2024-09-10T23:23:35.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.710+0000 7fbef9200640 1 -- 172.21.15.67:0/1790625651 wait complete. 2024-09-10T23:23:35.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.711+0000 7fbef9200640 1 Processor -- start 2024-09-10T23:23:35.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.711+0000 7fbef9200640 1 -- start start 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.711+0000 7fbef9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.711+0000 7fbef9200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 0x7fbef419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.711+0000 7fbef9200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef410b180 0x7fbef41a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef9200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4114c80 con 0x7fbef4104df0 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef9200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4114b00 con 0x7fbef410a800 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef9200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4114e00 con 0x7fbef410b180 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44718/0 (socket says 172.21.15.67:44718) 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 -- 172.21.15.67:0/952055696 learned_addr learned my addr 172.21.15.67:0/952055696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3e00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef410b180 0x7fbef41a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef2a00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 0x7fbef419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:35.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 -- 172.21.15.67:0/952055696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef410b180 msgr2=0x7fbef41a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef410b180 0x7fbef41a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 -- 172.21.15.67:0/952055696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 msgr2=0x7fbef419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 0x7fbef419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef3400640 1 -- 172.21.15.67:0/952055696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef41a11a0 con 0x7fbef4104df0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.712+0000 7fbef2a00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 0x7fbef419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbef3400640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbee400cce0 tx=0x7fbee400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee400f650 con 0x7fbef4104df0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef41a1490 con 0x7fbef4104df0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbee400fc80 con 0x7fbef4104df0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4004d20 con 0x7fbef4104df0 2024-09-10T23:23:35.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.713+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41a7b50 con 0x7fbef4104df0 2024-09-10T23:23:35.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.714+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef41051d0 con 0x7fbef4104df0 2024-09-10T23:23:35.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.716+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbee400f7f0 con 0x7fbef4104df0 2024-09-10T23:23:35.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.718+0000 7fbeebe00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 0x7fbed80b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:35.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.718+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7fbee40e6940 con 0x7fbef4104df0 2024-09-10T23:23:35.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.718+0000 7fbef2a00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 0x7fbed80b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:35.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.719+0000 7fbef2a00640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 0x7fbed80b79c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:35.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.722+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbee4014030 con 0x7fbef4104df0 2024-09-10T23:23:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.955+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef402e5c0 con 0x7fbef4104df0 2024-09-10T23:23:35.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.956+0000 7fbeebe00640 1 -- 172.21.15.67:0/952055696 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11906 (secure 0 0 0) 0x7fbee40ad020 con 0x7fbef4104df0 2024-09-10T23:23:35.958 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:23:35.958 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"btime":"2024-09-10T23:22:16:959905+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:22:16.055341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:23:35.958 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-09-10T23:23:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 msgr2=0x7fbed80b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 0x7fbed80b79c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0005a20 comp rx=0 tx=0).stop 2024-09-10T23:23:35.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 msgr2=0x7fbef419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbee400cce0 tx=0x7fbee400a620 comp rx=0 tx=0).stop 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 shutdown_connections 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fbed80b5500 0x7fbed80b79c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbef410b180 0x7fbef41a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbef410a800 0x7fbef419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 --2- 172.21.15.67:0/952055696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbef4104df0 0x7fbef419b0f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 >> 172.21.15.67:0/952055696 conn(0x7fbef4100840 msgr2=0x7fbef4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 shutdown_connections 2024-09-10T23:23:35.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:35.959+0000 7fbef9200640 1 -- 172.21.15.67:0/952055696 wait complete. 2024-09-10T23:23:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:36 smithi067 ceph-mon[30939]: pgmap v1163: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-09-10T23:23:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:36 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/952055696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:36 smithi145 ceph-mon[34397]: pgmap v1163: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-09-10T23:23:36.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:36 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/952055696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:36 smithi092 ceph-mon[34223]: pgmap v1163: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-09-10T23:23:36.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:36 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/952055696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:36.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id bb620b90 --release --await' 2024-09-10T23:23:36.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id bb620b90 --release --await 2024-09-10T23:23:37.207 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 -- 172.21.15.67:0/4243272970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c1011e0 msgr2=0x7f397c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 --2- 172.21.15.67:0/4243272970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c1011e0 0x7f397c1015c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f396c009900 tx=0x7f396c02a410 comp rx=0 tx=0).stop 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 -- 172.21.15.67:0/4243272970 shutdown_connections 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 --2- 172.21.15.67:0/4243272970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 --2- 172.21.15.67:0/4243272970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c105590 0x7f397c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 --2- 172.21.15.67:0/4243272970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c1011e0 0x7f397c1015c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 -- 172.21.15.67:0/4243272970 >> 172.21.15.67:0/4243272970 conn(0x7f397c078060 msgr2=0x7f397c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 -- 172.21.15.67:0/4243272970 shutdown_connections 2024-09-10T23:23:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.803+0000 7f3982a00640 1 -- 172.21.15.67:0/4243272970 wait complete. 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 Processor -- start 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 -- start start 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 0x7f397c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 0x7f397c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112cc0 con 0x7f397c105590 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112b40 con 0x7f397c1011e0 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.804+0000 7f3982a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112e40 con 0x7f397c105f10 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 0x7f397c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 0x7f397c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 0x7f397c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:38938/0 (socket says 172.21.15.67:38938) 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397be00640 1 -- 172.21.15.67:0/3757176916 learned_addr learned my addr 172.21.15.67:0/3757176916 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f3980a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 -- 172.21.15.67:0/3757176916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 msgr2=0x7f397c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 -- 172.21.15.67:0/3757176916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 msgr2=0x7f397c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 0x7f397c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 -- 172.21.15.67:0/3757176916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c1a1440 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f3980a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.805+0000 7f397b400640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 0x7f397c19b8d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3968005fd0 tx=0x7f396800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.806+0000 7f3978c00640 1 -- 172.21.15.67:0/3757176916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968009c30 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.806+0000 7f3978c00640 1 -- 172.21.15.67:0/3757176916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39680044c0 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.806+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c1a1730 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.806+0000 7f3978c00640 1 -- 172.21.15.67:0/3757176916 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968003d70 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.806+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1a7df0 con 0x7f397c105590 2024-09-10T23:23:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.809+0000 7f3985494740 1 Processor -- start 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.809+0000 7f3985494740 1 -- start start 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1f0a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1e75290 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1e646c0 con 0x55bdf1efe910 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3985494740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1e684d0 con 0x55bdf1efdff0 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3978c00640 1 -- 172.21.15.67:0/3757176916 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3968003f10 con 0x7f397c105590 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3980a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3980a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:37332/0 (socket says 172.21.15.67:37332) 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3980a00640 1 -- 172.21.15.67:0/4149151265 learned_addr learned my addr 172.21.15.67:0/4149151265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f397be00640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1f0a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f397b400640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3980a00640 1 -- 172.21.15.67:0/4149151265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 msgr2=0x55bdf1f0a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.810+0000 7f3980a00640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1f0a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3980a00640 1 -- 172.21.15.67:0/4149151265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 msgr2=0x55bdf1f0cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3980a00640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3980a00640 1 -- 172.21.15.67:0/4149151265 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf1df1460 con 0x55bdf1efdff0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f397b400640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3980a00640 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39740103d0 tx=0x7f3974010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd273b8660d06b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3971200640 1 -- 172.21.15.67:0/4149151265 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974011700 con 0x55bdf1efdff0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3971200640 1 -- 172.21.15.67:0/4149151265 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3974011d30 con 0x55bdf1efdff0 2024-09-10T23:23:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.811+0000 7f3971200640 1 -- 172.21.15.67:0/4149151265 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397401be10 con 0x55bdf1efdff0 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3978c00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 0x7f39600b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3978c00640 1 -- 172.21.15.67:0/3757176916 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f39680e5ff0 con 0x7f397c105590 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f397be00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 0x7f39600b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 -- 172.21.15.67:0/4149151265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 msgr2=0x55bdf1efe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39740103d0 tx=0x7f3974010890 comp rx=0 tx=0).stop 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 -- 172.21.15.67:0/4149151265 shutdown_connections 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1f0a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 --2- 172.21.15.67:0/4149151265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1efe3d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 -- 172.21.15.67:0/4149151265 >> 172.21.15.67:0/4149151265 conn(0x55bdf1efbe20 msgr2=0x55bdf1efcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 -- 172.21.15.67:0/4149151265 shutdown_connections 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.812+0000 7f3985494740 1 -- 172.21.15.67:0/4149151265 wait complete. 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.813+0000 7f397be00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 0x7f39600b7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f396c005ec0 tx=0x7f396c005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.813+0000 7f3985494740 1 -- start start 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.813+0000 7f3985494740 1 Processor -- start 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 0x55bdf1f078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efe910 0x55bdf1f07de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1cc5bb0 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1dff6c0 con 0x55bdf1efdff0 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bdf1f0d550 con 0x55bdf1efe910 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3985494740 10 client.0 _ll_register_callbacks cb 0x55bdf1e1a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44756/0 (socket says 172.21.15.67:44756) 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 -- 172.21.15.67:0/246093354 learned_addr learned my addr 172.21.15.67:0/246093354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f397be00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 0x55bdf1f078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f397b400640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efe910 0x55bdf1f07de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 -- 172.21.15.67:0/246093354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efe910 msgr2=0x55bdf1f07de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efe910 0x55bdf1f07de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 -- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 msgr2=0x55bdf1f078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 0x55bdf1f078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f3980a00640 1 -- 172.21.15.67:0/246093354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf1bc2b00 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.814+0000 7f397be00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 0x55bdf1f078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3980a00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39740099f0 tx=0x7f397401b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838be17c9ad435f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3965600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974029060 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3974011d30 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f3974011500 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3974030050 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3985494740 10 client.16479 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.815+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bdf1c39960 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3974011d30 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f39740055b0 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 10 client.16479 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 10 client.16479 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 20 client.16479 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f1a370 0x55bdf1f1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 4 client.16479 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf1e771e0 con 0x55bdf1f1a370 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.816+0000 7f397be00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f1a370 0x55bdf1f1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.817+0000 7f397be00640 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f1a370 0x55bdf1f1c760 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2239c673f5411e4a server_cookie=30f014597fd063de in_seq=0 out_seq=0 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.817+0000 7f3965600640 10 client.16479 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3965600640 1 -- 172.21.15.67:0/246093354 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bdf1e771e0 con 0x55bdf1f1a370 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3965600640 10 client.16479 handle_command_reply: tid=0 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 client.16479 shutdown 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 msgr2=0x55bdf1f08320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39740099f0 tx=0x7f397401b8b0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 shutdown_connections 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f1a370 0x55bdf1f1c760 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f08320 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efe910 0x55bdf1f07de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 --2- 172.21.15.67:0/246093354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efdff0 0x55bdf1f078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 >> 172.21.15.67:0/246093354 conn(0x55bdf1efbe20 msgr2=0x55bdf1f01990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3965600640 0 client.16479 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 shutdown_connections 2024-09-10T23:23:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.818+0000 7f3985494740 1 -- 172.21.15.67:0/246093354 wait complete. 2024-09-10T23:23:37.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.819+0000 7f3985494740 20 client.16479 trim_cache size 0 max 16384 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 Processor -- start 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 -- start start 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f02060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1efaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b4ae0 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b2350 con 0x55bdf1efe910 2024-09-10T23:23:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3985494740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b01c0 con 0x55bdf1efdff0 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3980a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3980a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44760/0 (socket says 172.21.15.67:44760) 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.837+0000 7f3980a00640 1 -- 172.21.15.67:0/1133905144 learned_addr learned my addr 172.21.15.67:0/1133905144 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f397be00640 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f02060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f397b400640 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1efaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 -- 172.21.15.67:0/1133905144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 msgr2=0x55bdf1f02060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f02060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 -- 172.21.15.67:0/1133905144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 msgr2=0x55bdf1efaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1efaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 -- 172.21.15.67:0/1133905144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf1efcb00 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3980a00640 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3974006010 tx=0x7f3974005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849b0881aa6c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3967e00640 1 -- 172.21.15.67:0/1133905144 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974029070 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3967e00640 1 -- 172.21.15.67:0/1133905144 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f397401b920 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.838+0000 7f3967e00640 1 -- 172.21.15.67:0/1133905144 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974005000 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 -- 172.21.15.67:0/1133905144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 msgr2=0x55bdf1f04ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3974006010 tx=0x7f3974005bf0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 -- 172.21.15.67:0/1133905144 shutdown_connections 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f04ad0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1efe910 0x55bdf1efaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 --2- 172.21.15.67:0/1133905144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f02060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 -- 172.21.15.67:0/1133905144 >> 172.21.15.67:0/1133905144 conn(0x55bdf1efbe20 msgr2=0x55bdf1efa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 -- 172.21.15.67:0/1133905144 shutdown_connections 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.839+0000 7f3985494740 1 -- 172.21.15.67:0/1133905144 wait complete. 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 -- start start 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 Processor -- start 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b0810 con 0x55bdf1efe910 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b1c80 con 0x55bdf1f0a8a0 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1b0ee0 con 0x55bdf1efdff0 2024-09-10T23:23:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.840+0000 7f3985494740 10 client.0 _ll_register_callbacks cb 0x7f396c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44772/0 (socket says 172.21.15.67:44772) 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 -- 172.21.15.67:0/2393740597 learned_addr learned my addr 172.21.15.67:0/2393740597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f3980a00640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397be00640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 -- 172.21.15.67:0/2393740597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 msgr2=0x55bdf1f129b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f129b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 -- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 msgr2=0x55bdf1f0f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 -- 172.21.15.67:0/2393740597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf1f0ffb0 con 0x55bdf1efe910 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f3980a00640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f397b400640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39680e3340 tx=0x7f3968003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c51cfe56d206ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.841+0000 7f3965600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968020040 con 0x55bdf1efe910 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9607+0+0 (secure 0 0 0) 0x7f39680e3560 con 0x55bdf1efe910 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397401b920 con 0x55bdf1efe910 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39680ab960 con 0x55bdf1efe910 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 10 client.16485 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bdf1f87a50 con 0x55bdf1efe910 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f397401b920 con 0x55bdf1efe910 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3968097190 con 0x55bdf1efe910 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 10 client.16485 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 10 client.16485 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.842+0000 7f3985494740 20 client.16485 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.843+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f88170 0x55bdf1f8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.843+0000 7f3985494740 4 client.16485 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb620b90", "release": true, "await": true}] 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.843+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb620b90", "release": true, "await": true}) -- 0x55bdf1e72c90 con 0x55bdf1f88170 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.843+0000 7f397be00640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f88170 0x55bdf1f8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.844+0000 7f397be00640 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f88170 0x55bdf1f8a590 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c629dba6b5c2c3 server_cookie=ead9d745dc24cf45 in_seq=0 out_seq=0 2024-09-10T23:23:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.844+0000 7f3965600640 10 client.16485 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.873+0000 7f3965600640 1 -- 172.21.15.67:0/2393740597 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bdf1e72c90 con 0x55bdf1f88170 2024-09-10T23:23:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.873+0000 7f3965600640 10 client.16485 handle_command_reply: tid=0 2024-09-10T23:23:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.873+0000 7f3985494740 1 client.16485 shutdown 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 msgr2=0x55bdf1f12f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39680e3340 tx=0x7f3968003820 comp rx=0 tx=0).stop 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 shutdown_connections 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55bdf1f88170 0x55bdf1f8a590 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bdf1f0a8a0 0x55bdf1f0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bdf1efe910 0x55bdf1f12f50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 --2- 172.21.15.67:0/2393740597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bdf1efdff0 0x55bdf1f129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 >> 172.21.15.67:0/2393740597 conn(0x55bdf1efbe20 msgr2=0x55bdf1efadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.874+0000 7f3965600640 0 client.16485 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.875+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 shutdown_connections 2024-09-10T23:23:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.875+0000 7f3985494740 1 -- 172.21.15.67:0/2393740597 wait complete. 2024-09-10T23:23:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.875+0000 7f3985494740 20 client.16485 trim_cache size 0 max 16384 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 37, 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 72, 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "bb620b90": { 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 72, 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:23:37.879 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.8, 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:37.881 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:23:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 msgr2=0x7f39600b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 0x7f39600b7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f396c005ec0 tx=0x7f396c005e50 comp rx=0 tx=0).stop 2024-09-10T23:23:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 msgr2=0x7f397c19b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 0x7f397c19b8d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3968005fd0 tx=0x7f396800dde0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 shutdown_connections 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39600b5450 0x7f39600b7910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f397c105f10 0x7f397c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f397c105590 0x7f397c19b8d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 --2- 172.21.15.67:0/3757176916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c1011e0 0x7f397c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.878+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 >> 172.21.15.67:0/3757176916 conn(0x7f397c078060 msgr2=0x7f397c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.879+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 shutdown_connections 2024-09-10T23:23:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:37.879+0000 7f3982a00640 1 -- 172.21.15.67:0/3757176916 wait complete. 2024-09-10T23:23:38.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:38 smithi145 ceph-mon[34397]: pgmap v1164: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.46k op/s 2024-09-10T23:23:38.385 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:38 smithi067 ceph-mon[30939]: pgmap v1164: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.46k op/s 2024-09-10T23:23:38.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:38 smithi092 ceph-mon[34223]: pgmap v1164: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.46k op/s 2024-09-10T23:23:38.928 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb620b90', total seconds elapsed: 59.0 2024-09-10T23:23:38.929 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-10T23:23:38.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:23:40.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:39 smithi092 ceph-mon[34223]: pgmap v1165: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:23:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:39 smithi092 ceph-mon[34223]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T23:23:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:39 smithi067 ceph-mon[30939]: pgmap v1165: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:23:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:39 smithi067 ceph-mon[30939]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T23:23:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:39 smithi145 ceph-mon[34397]: pgmap v1165: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:23:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:39 smithi145 ceph-mon[34397]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T23:23:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:42.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:41 smithi092 ceph-mon[34223]: pgmap v1167: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.9 MiB/s wr, 488 op/s 2024-09-10T23:23:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:41 smithi067 ceph-mon[30939]: pgmap v1167: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.9 MiB/s wr, 488 op/s 2024-09-10T23:23:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:41 smithi145 ceph-mon[34397]: pgmap v1167: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.9 MiB/s wr, 488 op/s 2024-09-10T23:23:44.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:43 smithi067 ceph-mon[30939]: pgmap v1168: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 585 op/s 2024-09-10T23:23:44.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:43 smithi145 ceph-mon[34397]: pgmap v1168: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 585 op/s 2024-09-10T23:23:44.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:43 smithi092 ceph-mon[34223]: pgmap v1168: 129 pgs: 129 active+clean; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 585 op/s 2024-09-10T23:23:46.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:45 smithi067 ceph-mon[30939]: pgmap v1169: 129 pgs: 129 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-10T23:23:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:45 smithi145 ceph-mon[34397]: pgmap v1169: 129 pgs: 129 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-10T23:23:46.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:45 smithi092 ceph-mon[34223]: pgmap v1169: 129 pgs: 129 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 1.10k op/s 2024-09-10T23:23:46.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 -- 172.21.15.67:0/64847863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a94101aa0 msgr2=0x7f1a94101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 --2- 172.21.15.67:0/64847863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a94101aa0 0x7f1a94101e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009860 tx=0x7f1a8802a340 comp rx=0 tx=0).stop 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 -- 172.21.15.67:0/64847863 shutdown_connections 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 --2- 172.21.15.67:0/64847863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a94106ee0 0x7f1a94107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 --2- 172.21.15.67:0/64847863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a941023c0 0x7f1a94106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 --2- 172.21.15.67:0/64847863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a94101aa0 0x7f1a94101e80 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009860 tx=0x7f1a8802a340 comp rx=0 tx=0).stop 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.604+0000 7f1a9b800640 1 -- 172.21.15.67:0/64847863 >> 172.21.15.67:0/64847863 conn(0x7f1a940fd730 msgr2=0x7f1a940ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.605+0000 7f1a9b800640 1 -- 172.21.15.67:0/64847863 shutdown_connections 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.605+0000 7f1a9b800640 1 -- 172.21.15.67:0/64847863 wait complete. 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.606+0000 7f1a9b800640 1 Processor -- start 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 -- start start 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 0x7f1a9406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a9406f130 0x7f1a9406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9406d2d0 con 0x7f1a9406f130 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9406d150 con 0x7f1a941023c0 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.607+0000 7f1a9b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9406d450 con 0x7f1a94106ee0 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.608+0000 7f1a93e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 0x7f1a9406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.608+0000 7f1a98e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:46.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.608+0000 7f1a98e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:47966/0 (socket says 172.21.15.67:47966) 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.608+0000 7f1a98e00640 1 -- 172.21.15.67:0/1798087770 learned_addr learned my addr 172.21.15.67:0/1798087770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a99800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a9406f130 0x7f1a9406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a98e00640 1 -- 172.21.15.67:0/1798087770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 msgr2=0x7f1a9406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a98e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 0x7f1a9406ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a98e00640 1 -- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a9406f130 msgr2=0x7f1a9406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a98e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a9406f130 0x7f1a9406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.609+0000 7f1a98e00640 1 -- 172.21.15.67:0/1798087770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a941b47f0 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.610+0000 7f1a93e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 0x7f1a9406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.611+0000 7f1a98e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a88035040 tx=0x7f1a880042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.612+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88040020 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.613+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a88032df0 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.613+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88037040 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.614+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a941b4a80 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.615+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941b4f10 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.616+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a94106890 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.627+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1a88071a70 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.629+0000 7f1a91600640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 0x7f1a7c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.630+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f1a88106060 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.630+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1a8804a070 con 0x7f1a941023c0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.631+0000 7f1a93e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 0x7f1a7c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:46.636+0000 7f1a93e00640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 0x7f1a7c0b7a10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005ab0 tx=0x7f1a800092a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:47.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.124+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a9406fef0 con 0x7f1a941023c0 2024-09-10T23:23:47.131 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:23:47.131 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"btime":"2024-09-10T23:22:16:959905+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:22:16.055341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.126+0000 7f1a91600640 1 -- 172.21.15.67:0/1798087770 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11906 (secure 0 0 0) 0x7f1a880cc820 con 0x7f1a941023c0 2024-09-10T23:23:47.132 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 msgr2=0x7f1a7c0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 0x7f1a7c0b7a10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005ab0 tx=0x7f1a800092a0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 msgr2=0x7f1a940749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a88035040 tx=0x7f1a880042f0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 shutdown_connections 2024-09-10T23:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f1a7c0b5550 0x7f1a7c0b7a10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a9406f130 0x7f1a9406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a94106ee0 0x7f1a9406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 --2- 172.21.15.67:0/1798087770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a941023c0 0x7f1a940749d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.132+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 >> 172.21.15.67:0/1798087770 conn(0x7f1a940fd730 msgr2=0x7f1a94110670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.133+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 shutdown_connections 2024-09-10T23:23:47.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:47.136+0000 7f1a9b800640 1 -- 172.21.15.67:0/1798087770 wait complete. 2024-09-10T23:23:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:47 smithi067 ceph-mon[30939]: pgmap v1170: 129 pgs: 129 active+clean; 14 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 1.77k op/s 2024-09-10T23:23:48.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:47 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1798087770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:47 smithi145 ceph-mon[34397]: pgmap v1170: 129 pgs: 129 active+clean; 14 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 1.77k op/s 2024-09-10T23:23:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:47 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1798087770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:47 smithi092 ceph-mon[34223]: pgmap v1170: 129 pgs: 129 active+clean; 14 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 1.77k op/s 2024-09-10T23:23:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:47 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1798087770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:23:50.133 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:49 smithi145 ceph-mon[34397]: pgmap v1171: 129 pgs: 129 active+clean; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T23:23:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:49 smithi092 ceph-mon[34223]: pgmap v1171: 129 pgs: 129 active+clean; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T23:23:50.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:49 smithi067 ceph-mon[30939]: pgmap v1171: 129 pgs: 129 active+clean; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T23:23:52.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:51 smithi067 ceph-mon[30939]: pgmap v1172: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:51 smithi145 ceph-mon[34397]: pgmap v1172: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:52.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:51 smithi092 ceph-mon[34223]: pgmap v1172: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:52.427 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-10T23:23:52.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-10T23:23:53.633 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:23:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:53 smithi145 ceph-mon[34397]: pgmap v1173: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:54.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:53 smithi092 ceph-mon[34223]: pgmap v1173: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:54.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:53 smithi067 ceph-mon[30939]: pgmap v1173: 129 pgs: 129 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 2.67k op/s 2024-09-10T23:23:56.117 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:55 smithi067 ceph-mon[30939]: pgmap v1174: 129 pgs: 129 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:23:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:55 smithi145 ceph-mon[34397]: pgmap v1174: 129 pgs: 129 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:23:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:55 smithi092 ceph-mon[34223]: pgmap v1174: 129 pgs: 129 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:23:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.367+0000 7f85e7e00640 1 -- 172.21.15.67:0/3238524482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 msgr2=0x7f85e810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.367+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3238524482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e810da10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009900 tx=0x7f85d802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 -- 172.21.15.67:0/3238524482 shutdown_connections 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3238524482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8113970 0x7f85e811a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3238524482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8112ff0 0x7f85e8113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3238524482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e810da10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 -- 172.21.15.67:0/3238524482 >> 172.21.15.67:0/3238524482 conn(0x7f85e806c950 msgr2=0x7f85e806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 -- 172.21.15.67:0/3238524482 shutdown_connections 2024-09-10T23:23:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.371+0000 7f85e7e00640 1 -- 172.21.15.67:0/3238524482 wait complete. 2024-09-10T23:23:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.372+0000 7f85e7e00640 1 Processor -- start 2024-09-10T23:23:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 -- start start 2024-09-10T23:23:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e81a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8113970 0x7f85e81a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85e811d480 con 0x7f85e810d630 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e811d300 con 0x7f85e8113970 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.375+0000 7f85e7e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85e811d600 con 0x7f85e8112ff0 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.376+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e81a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.376+0000 7f85e6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.376+0000 7f85e6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:58010/0 (socket says 172.21.15.67:58010) 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.376+0000 7f85e6000640 1 -- 172.21.15.67:0/3416322698 learned_addr learned my addr 172.21.15.67:0/3416322698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.376+0000 7f85e7400640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8113970 0x7f85e81a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.377+0000 7f85e6000640 1 -- 172.21.15.67:0/3416322698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8113970 msgr2=0x7f85e81a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.377+0000 7f85e6000640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8113970 0x7f85e81a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.377+0000 7f85e6000640 1 -- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 msgr2=0x7f85e81a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.377+0000 7f85e6000640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e81a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.377+0000 7f85e6000640 1 -- 172.21.15.67:0/3416322698 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81a99a0 con 0x7f85e8112ff0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.378+0000 7f85e6000640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c920 tx=0x7f85d400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.378+0000 7f85eca00640 1 -- 172.21.15.67:0/3416322698 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007c70 con 0x7f85e8112ff0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.379+0000 7f85eca00640 1 -- 172.21.15.67:0/3416322698 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d4007e10 con 0x7f85e8112ff0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.379+0000 7f85eca00640 1 -- 172.21.15.67:0/3416322698 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018710 con 0x7f85e8112ff0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.379+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81a9c90 con 0x7f85e8112ff0 2024-09-10T23:23:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.379+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e81b0350 con 0x7f85e8112ff0 2024-09-10T23:23:57.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.387+0000 7f85eca00640 1 -- 172.21.15.67:0/3416322698 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f85d400f450 con 0x7f85e8112ff0 2024-09-10T23:23:57.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.392+0000 7f85eca00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 0x7f85cc0b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.393+0000 7f85eca00640 1 -- 172.21.15.67:0/3416322698 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f85d40e64c0 con 0x7f85e8112ff0 2024-09-10T23:23:57.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.393+0000 7f85e6a00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 0x7f85cc0b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.395+0000 7f85e6a00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 0x7f85cc0b7b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006010 tx=0x7f85d8004300 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.410+0000 7f85f0828740 1 Processor -- start 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.413+0000 7f85f0828740 1 -- start start 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557909a8aa90 con 0x557909b1e0c0 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557909a79ec0 con 0x557909b1f620 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.414+0000 7f85f0828740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557909a7dcd0 con 0x557909b29ef0 2024-09-10T23:23:57.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.415+0000 7f85e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.415+0000 7f85e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48190/0 (socket says 172.21.15.67:48190) 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.415+0000 7f85e6000640 1 -- 172.21.15.67:0/2461196887 learned_addr learned my addr 172.21.15.67:0/2461196887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.416+0000 7f85e6000640 1 -- 172.21.15.67:0/2461196887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 msgr2=0x557909b2c2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.416+0000 7f85e6000640 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b2c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.416+0000 7f85e6000640 1 -- 172.21.15.67:0/2461196887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 msgr2=0x557909b1db80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.416+0000 7f85e6000640 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b1db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.416+0000 7f85e6000640 1 -- 172.21.15.67:0/2461196887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557909a1f9d0 con 0x557909b1e0c0 2024-09-10T23:23:57.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.418+0000 7f85e6000640 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85d40ac4f0 tx=0x7f85d40ac070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0b2f9b2a9ad08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.419+0000 7f85d3e00640 1 -- 172.21.15.67:0/2461196887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40b1040 con 0x557909b1e0c0 2024-09-10T23:23:57.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.419+0000 7f85d3e00640 1 -- 172.21.15.67:0/2461196887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40b5af0 con 0x557909b1e0c0 2024-09-10T23:23:57.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.419+0000 7f85d3e00640 1 -- 172.21.15.67:0/2461196887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40a87d0 con 0x557909b1e0c0 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 -- 172.21.15.67:0/2461196887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 msgr2=0x557909b1e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85d40ac4f0 tx=0x7f85d40ac070 comp rx=0 tx=0).stop 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 -- 172.21.15.67:0/2461196887 shutdown_connections 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1e4e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 --2- 172.21.15.67:0/2461196887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b1db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.421+0000 7f85f0828740 1 -- 172.21.15.67:0/2461196887 >> 172.21.15.67:0/2461196887 conn(0x557909adf6f0 msgr2=0x557909b1b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:57.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.422+0000 7f85f0828740 1 -- 172.21.15.67:0/2461196887 shutdown_connections 2024-09-10T23:23:57.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.422+0000 7f85f0828740 1 -- 172.21.15.67:0/2461196887 wait complete. 2024-09-10T23:23:57.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.423+0000 7f85f0828740 1 -- start start 2024-09-10T23:23:57.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.424+0000 7f85f0828740 1 Processor -- start 2024-09-10T23:23:57.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.425+0000 7f85f0828740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.426+0000 7f85f0828740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.426+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.427+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:43774/0 (socket says 172.21.15.67:43774) 2024-09-10T23:23:57.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.427+0000 7f85e6a00640 1 -- 172.21.15.67:0/2691690057 learned_addr learned my addr 172.21.15.67:0/2691690057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:57.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.427+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.427+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5579098db7b0 con 0x557909b1f620 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.428+0000 7f85e7400640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.428+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557909a1e650 con 0x557909b1e0c0 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.428+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557909b12f20 con 0x557909b29ef0 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.429+0000 7f85f0828740 10 client.0 _ll_register_callbacks cb 0x5579097850a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e6000640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e7400640 1 -- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 msgr2=0x557909b26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e7400640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e7400640 1 -- 172.21.15.67:0/2691690057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 msgr2=0x557909b274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e7400640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.430+0000 7f85e7400640 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579097e7ba0 con 0x557909b29ef0 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.431+0000 7f85e6a00640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.431+0000 7f85e6000640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:23:57.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.431+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d40b5af0 con 0x557909b29ef0 2024-09-10T23:23:57.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.432+0000 7f85d0c00640 10 client.26179 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:23:57.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.432+0000 7f85e7400640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b27a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f9e0 tx=0x7f85e000fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97da6ffb2c747882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.433+0000 7f85d0c00640 1 -- 172.21.15.67:0/2691690057 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004d20 con 0x557909b29ef0 2024-09-10T23:23:57.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.433+0000 7f85d0c00640 1 -- 172.21.15.67:0/2691690057 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f85e0004b20 con 0x557909b29ef0 2024-09-10T23:23:57.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.433+0000 7f85d0c00640 1 -- 172.21.15.67:0/2691690057 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f85e0004d20 con 0x557909b29ef0 2024-09-10T23:23:57.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.435+0000 7f85f0828740 10 client.26179 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:57.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.435+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({fsmap=37}) -- 0x557909b410c0 con 0x557909b29ef0 2024-09-10T23:23:57.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.435+0000 7f85d0c00640 1 -- 172.21.15.67:0/2691690057 <== mon.1 v2:172.21.15.145:3300/0 4 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f85e000db40 con 0x557909b29ef0 2024-09-10T23:23:57.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.435+0000 7f85f0828740 10 client.26179 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:57.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85f0828740 10 client.26179 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:57.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85f0828740 20 client.26179 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:57.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909b424e0 0x557909b44930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85f0828740 4 client.26179 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:23:57.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557909a8c9e0 con 0x557909b424e0 2024-09-10T23:23:57.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.436+0000 7f85e6a00640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909b424e0 0x557909b44930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.437+0000 7f85e6a00640 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909b424e0 0x557909b44930 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b72399de2b322a server_cookie=17bb1b0d9aee1465 in_seq=0 out_seq=0 2024-09-10T23:23:57.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.438+0000 7f85d0c00640 10 client.26179 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:57.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.438+0000 7f85d0c00640 1 -- 172.21.15.67:0/2691690057 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557909a8c9e0 con 0x557909b424e0 2024-09-10T23:23:57.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.438+0000 7f85d0c00640 10 client.26179 handle_command_reply: tid=0 2024-09-10T23:23:57.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 client.26179 shutdown 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 msgr2=0x557909b27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b27a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f9e0 tx=0x7f85e000fea0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 shutdown_connections 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909b424e0 0x557909b44930 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85d0c00640 0 client.26179 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.439+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b27a10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 1 --2- 172.21.15.67:0/2691690057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1e0c0 0x557909b26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 >> 172.21.15.67:0/2691690057 conn(0x557909adf6f0 msgr2=0x557909b122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:57.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 shutdown_connections 2024-09-10T23:23:57.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 1 -- 172.21.15.67:0/2691690057 wait complete. 2024-09-10T23:23:57.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.440+0000 7f85f0828740 20 client.26179 trim_cache size 0 max 16384 2024-09-10T23:23:57.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.484+0000 7f85f0828740 1 Processor -- start 2024-09-10T23:23:57.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.485+0000 7f85f0828740 1 -- start start 2024-09-10T23:23:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.485+0000 7f85f0828740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.485+0000 7f85f0828740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.485+0000 7f85f0828740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b246e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.486+0000 7f85f0828740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b8b60 con 0x557909b1e0c0 2024-09-10T23:23:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.486+0000 7f85f0828740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b63d0 con 0x557909b1f620 2024-09-10T23:23:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.486+0000 7f85f0828740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b4260 con 0x557909b29ef0 2024-09-10T23:23:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.487+0000 7f85e6000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.487+0000 7f85e6000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:43788/0 (socket says 172.21.15.67:43788) 2024-09-10T23:23:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.487+0000 7f85e6000640 1 -- 172.21.15.67:0/981386744 learned_addr learned my addr 172.21.15.67:0/981386744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.488+0000 7f85e7400640 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e7400640 1 -- 172.21.15.67:0/981386744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 msgr2=0x557909b241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e7400640 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e7400640 1 -- 172.21.15.67:0/981386744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 msgr2=0x557909b1d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e7400640 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e7400640 1 -- 172.21.15.67:0/981386744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557909b1edb0 con 0x557909b29ef0 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.489+0000 7f85e6000640 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:23:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.490+0000 7f85e7400640 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b246e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f9e0 tx=0x7f85e0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a2af899ada3176c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.490+0000 7f85d3400640 1 -- 172.21.15.67:0/981386744 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0028040 con 0x557909b29ef0 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.490+0000 7f85d3400640 1 -- 172.21.15.67:0/981386744 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40b5af0 con 0x557909b29ef0 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.490+0000 7f85d3400640 1 -- 172.21.15.67:0/981386744 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004280 con 0x557909b29ef0 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 -- 172.21.15.67:0/981386744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 msgr2=0x557909b246e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b246e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f9e0 tx=0x7f85e0012000 comp rx=0 tx=0).stop 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 -- 172.21.15.67:0/981386744 shutdown_connections 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b29ef0 0x557909b246e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b1f620 0x557909b241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 --2- 172.21.15.67:0/981386744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1e0c0 0x557909b1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.491+0000 7f85f0828740 1 -- 172.21.15.67:0/981386744 >> 172.21.15.67:0/981386744 conn(0x557909adf6f0 msgr2=0x557909b28de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.492+0000 7f85f0828740 1 -- 172.21.15.67:0/981386744 shutdown_connections 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.492+0000 7f85f0828740 1 -- 172.21.15.67:0/981386744 wait complete. 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.492+0000 7f85f0828740 1 -- start start 2024-09-10T23:23:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.493+0000 7f85f0828740 1 Processor -- start 2024-09-10T23:23:57.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.493+0000 7f85f0828740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 0x557909b18990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.493+0000 7f85f0828740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b16460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.494+0000 7f85f0828740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 0x557909b169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.494+0000 7f85f0828740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b48b0 con 0x557909b1f620 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.494+0000 7f85f0828740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b5d00 con 0x557909b29ef0 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.494+0000 7f85f0828740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b4f60 con 0x557909b1e0c0 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85f0828740 10 client.0 _ll_register_callbacks cb 0x7f85e000a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.494+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 0x557909b18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 0x557909b18990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:58044/0 (socket says 172.21.15.67:58044) 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e6a00640 1 -- 172.21.15.67:0/686293004 learned_addr learned my addr 172.21.15.67:0/686293004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 0x557909b169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 -- 172.21.15.67:0/686293004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 msgr2=0x557909b18990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 0x557909b18990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 -- 172.21.15.67:0/686293004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 msgr2=0x557909b16460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b16460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:57.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.495+0000 7f85e7400640 1 -- 172.21.15.67:0/686293004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557909b171b0 con 0x557909b29ef0 2024-09-10T23:23:57.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.496+0000 7f85e7400640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 0x557909b169a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f85e0002230 tx=0x7f85e00059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c20722a1bcdd2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:23:57.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.496+0000 7f85d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:23:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.497+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e003a060 con 0x557909b29ef0 2024-09-10T23:23:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.497+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d40b5af0 con 0x557909b29ef0 2024-09-10T23:23:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.497+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f85e0035470 con 0x557909b29ef0 2024-09-10T23:23:57.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.498+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f85e0028070 con 0x557909b29ef0 2024-09-10T23:23:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.498+0000 7f85f0828740 10 client.36221 fetch_fsmap learned FSMap version 37 2024-09-10T23:23:57.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.498+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557909bb0440 con 0x557909b29ef0 2024-09-10T23:23:57.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.499+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40b5af0 con 0x557909b29ef0 2024-09-10T23:23:57.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.500+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f85e00356b0 con 0x557909b29ef0 2024-09-10T23:23:57.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.500+0000 7f85f0828740 10 client.36221 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:23:57.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.500+0000 7f85f0828740 10 client.36221 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:23:57.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.500+0000 7f85f0828740 20 client.36221 populate_metadata read hostname 'smithi067' 2024-09-10T23:23:57.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.500+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909bb0b20 0x557909bb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:23:57.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.501+0000 7f85f0828740 4 client.36221 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-10T23:23:57.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.501+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x557909a88490 con 0x557909bb0b20 2024-09-10T23:23:57.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.502+0000 7f85e6a00640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909bb0b20 0x557909bb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:23:57.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.505+0000 7f85d0c00640 10 client.36221 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:57.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:57.505+0000 7f85e6a00640 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909bb0b20 0x557909bb2f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826a73212bb17bdb server_cookie=3fad5d1f9ec50fde in_seq=0 out_seq=0 2024-09-10T23:23:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:57 smithi145 ceph-mon[34397]: pgmap v1175: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T23:23:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:57 smithi092 ceph-mon[34223]: pgmap v1175: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T23:23:58.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:57 smithi067 ceph-mon[30939]: pgmap v1175: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T23:23:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85d0c00640 1 -- 172.21.15.67:0/686293004 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557909a88490 con 0x557909bb0b20 2024-09-10T23:23:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85d0c00640 10 client.36221 handle_command_reply: tid=0 2024-09-10T23:23:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 client.36221 shutdown 2024-09-10T23:23:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 msgr2=0x557909b169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 0x557909b169a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f85e0002230 tx=0x7f85e00059b0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 shutdown_connections 2024-09-10T23:23:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557909bb0b20 0x557909bb2f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557909b29ef0 0x557909b169a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.550+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557909b1f620 0x557909b16460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85f0828740 1 --2- 172.21.15.67:0/686293004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557909b1e0c0 0x557909b18990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 >> 172.21.15.67:0/686293004 conn(0x557909adf6f0 msgr2=0x557909b11aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85d0c00640 0 client.36221 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 shutdown_connections 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85f0828740 1 -- 172.21.15.67:0/686293004 wait complete. 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.551+0000 7f85f0828740 20 client.36221 trim_cache size 0 max 16384 2024-09-10T23:23:59.553 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 37, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 74, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "1d31018d": { 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 74, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:23:59.554 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.8, 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:23:59.555 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 msgr2=0x7f85cc0b7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 0x7f85cc0b7b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006010 tx=0x7f85d8004300 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 msgr2=0x7f85e81a3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c920 tx=0x7f85d400cde0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 shutdown_connections 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f85cc0b56a0 0x7f85cc0b7b60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e8113970 0x7f85e81a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e8112ff0 0x7f85e81a3e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3416322698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e810d630 0x7f85e81a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:23:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.552+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 >> 172.21.15.67:0/3416322698 conn(0x7f85e806c950 msgr2=0x7f85e810fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:23:59.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.553+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 shutdown_connections 2024-09-10T23:23:59.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:23:59.553+0000 7f85e7e00640 1 -- 172.21.15.67:0/3416322698 wait complete. 2024-09-10T23:23:59.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:23:59 smithi067 ceph-mon[30939]: pgmap v1176: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:24:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:23:59 smithi145 ceph-mon[34397]: pgmap v1176: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:24:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:23:59 smithi092 ceph-mon[34223]: pgmap v1176: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 3.09k op/s 2024-09-10T23:24:00.466 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d31018d', quiesce took 21.5 seconds. Will release after: -2.6499999999999986 2024-09-10T23:24:00.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:24:00.855 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:24:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:01 smithi145 ceph-mon[34397]: pgmap v1177: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:02.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:01 smithi092 ceph-mon[34223]: pgmap v1177: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:01 smithi067 ceph-mon[30939]: pgmap v1177: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:02.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 -- 172.21.15.67:0/2795752514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb41045c0 msgr2=0x7f7cb410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2795752514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb41045c0 0x7f7cb410cab0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009900 tx=0x7f7ca002a400 comp rx=0 tx=0).stop 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 -- 172.21.15.67:0/2795752514 shutdown_connections 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2795752514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb410d100 0x7f7cb410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2795752514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb41045c0 0x7f7cb410cab0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2795752514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb4103ca0 0x7f7cb4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 -- 172.21.15.67:0/2795752514 >> 172.21.15.67:0/2795752514 conn(0x7f7cb40fd700 msgr2=0x7f7cb40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 -- 172.21.15.67:0/2795752514 shutdown_connections 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.467+0000 7f7cbc200640 1 -- 172.21.15.67:0/2795752514 wait complete. 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.468+0000 7f7cbc200640 1 Processor -- start 2024-09-10T23:24:02.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.468+0000 7f7cbc200640 1 -- start start 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.468+0000 7f7cbc200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cbc200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb41045c0 0x7f7cb419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cbc200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb410d100 0x7f7cb41a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cbc200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4112c70 con 0x7f7cb4103ca0 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cbc200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4112af0 con 0x7f7cb41045c0 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cbc200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4112df0 con 0x7f7cb410d100 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32824/0 (socket says 172.21.15.67:32824) 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 -- 172.21.15.67:0/2209457855 learned_addr learned my addr 172.21.15.67:0/2209457855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb8e00640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb41045c0 0x7f7cb419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cba200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb410d100 0x7f7cb41a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 -- 172.21.15.67:0/2209457855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb410d100 msgr2=0x7f7cb41a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb410d100 0x7f7cb41a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 -- 172.21.15.67:0/2209457855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb41045c0 msgr2=0x7f7cb419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb41045c0 0x7f7cb419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.469+0000 7f7cb9800640 1 -- 172.21.15.67:0/2209457855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb41a14b0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cb9800640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800d950 tx=0x7f7ca800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017070 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ca8009c30 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800f620 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41a17a0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.470+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a7e60 con 0x7f7cb4103ca0 2024-09-10T23:24:02.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.471+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb4104080 con 0x7f7cb4103ca0 2024-09-10T23:24:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.474+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7ca8010460 con 0x7f7cb4103ca0 2024-09-10T23:24:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.476+0000 7f7cb2000640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 0x7f7c9c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.476+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f7ca80e5fb0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.476+0000 7f7cb8e00640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 0x7f7c9c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.477+0000 7f7cb8e00640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 0x7f7c9c0b7910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00057c0 tx=0x7f7ca0005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.480+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7ca8009de0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.719+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cb41a81a0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.720+0000 7f7cb2000640 1 -- 172.21.15.67:0/2209457855 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11906 (secure 0 0 0) 0x7f7ca80ac0a0 con 0x7f7cb4103ca0 2024-09-10T23:24:02.722 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:24:02.722 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"btime":"2024-09-10T23:22:16:959905+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:22:16.055341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:24:02.722 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-09-10T23:24:02.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 msgr2=0x7f7c9c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:02.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 0x7f7c9c0b7910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00057c0 tx=0x7f7ca0005e80 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 msgr2=0x7f7cb419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800d950 tx=0x7f7ca800de10 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 shutdown_connections 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c9c0b5450 0x7f7c9c0b7910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cb410d100 0x7f7cb41a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb41045c0 0x7f7cb419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 --2- 172.21.15.67:0/2209457855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cb4103ca0 0x7f7cb419b400 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 >> 172.21.15.67:0/2209457855 conn(0x7f7cb40fd700 msgr2=0x7f7cb40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 shutdown_connections 2024-09-10T23:24:02.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:02.723+0000 7f7cbc200640 1 -- 172.21.15.67:0/2209457855 wait complete. 2024-09-10T23:24:03.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:02 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2209457855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:24:03.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:02 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2209457855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:24:03.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:02 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2209457855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:24:03.525 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 1d31018d --release --await' 2024-09-10T23:24:03.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 1d31018d --release --await 2024-09-10T23:24:03.920 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:24:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:03 smithi145 ceph-mon[34397]: pgmap v1178: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-10T23:24:04.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:03 smithi067 ceph-mon[30939]: pgmap v1178: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-10T23:24:04.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:03 smithi092 ceph-mon[34223]: pgmap v1178: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-09-10T23:24:04.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 -- 172.21.15.67:0/2795275895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 msgr2=0x7f297810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 --2- 172.21.15.67:0/2795275895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297810cad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2960009900 tx=0x7f296002a410 comp rx=0 tx=0).stop 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 -- 172.21.15.67:0/2795275895 shutdown_connections 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 --2- 172.21.15.67:0/2795275895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 0x7f297810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 --2- 172.21.15.67:0/2795275895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297810cad0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 --2- 172.21.15.67:0/2795275895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f29781040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 -- 172.21.15.67:0/2795275895 >> 172.21.15.67:0/2795275895 conn(0x7f29780fd700 msgr2=0x7f29780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 -- 172.21.15.67:0/2795275895 shutdown_connections 2024-09-10T23:24:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.604+0000 7f297f400640 1 -- 172.21.15.67:0/2795275895 wait complete. 2024-09-10T23:24:04.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.605+0000 7f297f400640 1 Processor -- start 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.605+0000 7f297f400640 1 -- start start 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.605+0000 7f297f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f297819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.605+0000 7f297f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.605+0000 7f297f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 0x7f29781a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2978112c90 con 0x7f29781045e0 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297f400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2978112b10 con 0x7f2978103cc0 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2978112e10 con 0x7f297810d120 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32844/0 (socket says 172.21.15.67:32844) 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 -- 172.21.15.67:0/4147660734 learned_addr learned my addr 172.21.15.67:0/4147660734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297d400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 0x7f29781a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297ca00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f297819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 -- 172.21.15.67:0/4147660734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 msgr2=0x7f29781a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 0x7f29781a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 -- 172.21.15.67:0/4147660734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 msgr2=0x7f297819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f297819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f296fe00640 1 -- 172.21.15.67:0/4147660734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781a1470 con 0x7f29781045e0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.606+0000 7f297ca00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f297819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f296fe00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f296002a930 tx=0x7f2960003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f2976000640 1 -- 172.21.15.67:0/4147660734 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296003f070 con 0x7f29781045e0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f2976000640 1 -- 172.21.15.67:0/4147660734 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296003baf0 con 0x7f29781045e0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29781a1700 con 0x7f29781045e0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f2976000640 1 -- 172.21.15.67:0/4147660734 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960035030 con 0x7f29781045e0 2024-09-10T23:24:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.607+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781a7e20 con 0x7f29781045e0 2024-09-10T23:24:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 Processor -- start 2024-09-10T23:24:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 -- start start 2024-09-10T23:24:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2976000640 1 -- 172.21.15.67:0/4147660734 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f296003b660 con 0x7f29781045e0 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920f98a0 0x5573920fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557392064290 con 0x5573920ecff0 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573920536c0 con 0x5573920ed910 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f2981fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5573920574d0 con 0x5573920f98a0 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.611+0000 7f297d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297ca00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32852/0 (socket says 172.21.15.67:32852) 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297ca00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:58732/0 (socket says 172.21.15.67:58732) 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 -- 172.21.15.67:0/1612743216 learned_addr learned my addr 172.21.15.67:0/1612743216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f296fe00640 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920f98a0 0x5573920fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 -- 172.21.15.67:0/1612743216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920f98a0 msgr2=0x5573920fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920f98a0 0x5573920fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 -- 172.21.15.67:0/1612743216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 msgr2=0x5573920f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 -- 172.21.15.67:0/1612743216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557391fe0460 con 0x5573920ecff0 2024-09-10T23:24:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.612+0000 7f297d400640 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f29700103a0 tx=0x7f2970010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a97daf285030aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f296dc00640 1 -- 172.21.15.67:0/1612743216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970016070 con 0x5573920ecff0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f296dc00640 1 -- 172.21.15.67:0/1612743216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2970004d20 con 0x5573920ecff0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f296dc00640 1 -- 172.21.15.67:0/1612743216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970005390 con 0x5573920ecff0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f2976000640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 0x7f295c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f2976000640 1 -- 172.21.15.67:0/4147660734 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f2960106e00 con 0x7f29781045e0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f297ca00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 0x7f295c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f2981fde740 1 -- 172.21.15.67:0/1612743216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 msgr2=0x5573920ed3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.613+0000 7f2981fde740 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f29700103a0 tx=0x7f2970010860 comp rx=0 tx=0).stop 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 -- 172.21.15.67:0/1612743216 shutdown_connections 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920f98a0 0x5573920fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 --2- 172.21.15.67:0/1612743216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920ed3d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 -- 172.21.15.67:0/1612743216 >> 172.21.15.67:0/1612743216 conn(0x5573920eae20 msgr2=0x5573920ebfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 -- 172.21.15.67:0/1612743216 shutdown_connections 2024-09-10T23:24:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 -- 172.21.15.67:0/1612743216 wait complete. 2024-09-10T23:24:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f297ca00640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 0x7f295c0b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5573920f0170 tx=0x7f29680084c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.614+0000 7f2981fde740 1 -- start start 2024-09-10T23:24:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 Processor -- start 2024-09-10T23:24:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ecff0 0x5573920f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557391eb4bb0 con 0x5573920f98a0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557391fee6c0 con 0x5573920ed910 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5573920fc550 con 0x5573920ecff0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f2981fde740 10 client.0 _ll_register_callbacks cb 0x557392009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f297d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f296fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f297d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32868/0 (socket says 172.21.15.67:32868) 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.615+0000 7f297d400640 1 -- 172.21.15.67:0/3536763581 learned_addr learned my addr 172.21.15.67:0/3536763581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f297ca00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ecff0 0x5573920f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 -- 172.21.15.67:0/3536763581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ecff0 msgr2=0x5573920f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ecff0 0x5573920f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 -- 172.21.15.67:0/3536763581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 msgr2=0x5573920f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 -- 172.21.15.67:0/3536763581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557391db1b00 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f297d400640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f296fe00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f6e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2960104110 tx=0x7f296003ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73094e8983e2ce7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f2966000640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960049070 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2970004d20 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.616+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f29600b40e0 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f29600c8ce0 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 10 client.36236 fetch_fsmap learned FSMap version 37 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557391e28960 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2970004d20 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f2960104390 con 0x5573920ed910 2024-09-10T23:24:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 10 client.36236 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 10 client.36236 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 20 client.36236 populate_metadata read hostname 'smithi067' 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921094c0 0x55739210b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 4 client.36236 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.617+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573920661e0 con 0x5573921094c0 2024-09-10T23:24:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.618+0000 7f297ca00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921094c0 0x55739210b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.619+0000 7f297ca00640 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921094c0 0x55739210b8d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2878d26906993a2a server_cookie=d73a82a327c6db06 in_seq=0 out_seq=0 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.619+0000 7f2966000640 10 client.36236 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2966000640 1 -- 172.21.15.67:0/3536763581 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573920661e0 con 0x5573921094c0 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2966000640 10 client.36236 handle_command_reply: tid=0 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 client.36236 shutdown 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 msgr2=0x5573920f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f6e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2960104110 tx=0x7f296003ba40 comp rx=0 tx=0).stop 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 shutdown_connections 2024-09-10T23:24:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921094c0 0x55739210b8d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920f98a0 0x5573920f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920ed910 0x5573920f6e00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 --2- 172.21.15.67:0/3536763581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ecff0 0x5573920f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 >> 172.21.15.67:0/3536763581 conn(0x5573920eae20 msgr2=0x5573920f09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2966000640 0 client.36236 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 shutdown_connections 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 1 -- 172.21.15.67:0/3536763581 wait complete. 2024-09-10T23:24:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.620+0000 7f2981fde740 20 client.36236 trim_cache size 0 max 16384 2024-09-10T23:24:04.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.638+0000 7f2981fde740 1 Processor -- start 2024-09-10T23:24:04.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.638+0000 7f2981fde740 1 -- start start 2024-09-10T23:24:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920f4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b49a0 con 0x5573920ecff0 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b2210 con 0x5573920f98a0 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f2981fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b0080 con 0x5573920ed910 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32874/0 (socket says 172.21.15.67:32874) 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 -- 172.21.15.67:0/3517479129 learned_addr learned my addr 172.21.15.67:0/3517479129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f296fe00640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297d400640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 -- 172.21.15.67:0/3517479129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 msgr2=0x5573920f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 -- 172.21.15.67:0/3517479129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 msgr2=0x5573921111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.639+0000 7f297ca00640 1 -- 172.21.15.67:0/3517479129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739210bf50 con 0x5573920ecff0 2024-09-10T23:24:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.640+0000 7f297d400640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:24:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.640+0000 7f297ca00640 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55739210c180 tx=0x7f2968014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eb3fa7459683b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.640+0000 7f296d200640 1 -- 172.21.15.67:0/3517479129 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968003bd0 con 0x5573920ecff0 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.640+0000 7f296d200640 1 -- 172.21.15.67:0/3517479129 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2968003d70 con 0x5573920ecff0 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.640+0000 7f296d200640 1 -- 172.21.15.67:0/3517479129 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296801c670 con 0x5573920ecff0 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- 172.21.15.67:0/3517479129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 msgr2=0x5573920f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55739210c180 tx=0x7f2968014000 comp rx=0 tx=0).stop 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- 172.21.15.67:0/3517479129 shutdown_connections 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 --2- 172.21.15.67:0/3517479129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920f3ad0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- 172.21.15.67:0/3517479129 >> 172.21.15.67:0/3517479129 conn(0x5573920eae20 msgr2=0x5573920e9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- 172.21.15.67:0/3517479129 shutdown_connections 2024-09-10T23:24:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- 172.21.15.67:0/3517479129 wait complete. 2024-09-10T23:24:04.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.641+0000 7f2981fde740 1 -- start start 2024-09-10T23:24:04.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 Processor -- start 2024-09-10T23:24:04.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b06d0 con 0x5573920ecff0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b1b40 con 0x5573920f98a0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b0da0 con 0x5573920ed910 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f2981fde740 10 client.0 _ll_register_callbacks cb 0x7f296010a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f297ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f297ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32878/0 (socket says 172.21.15.67:32878) 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.642+0000 7f297ca00640 1 -- 172.21.15.67:0/1057989370 learned_addr learned my addr 172.21.15.67:0/1057989370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f296fe00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297d400640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 -- 172.21.15.67:0/1057989370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 msgr2=0x5573920fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 -- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 msgr2=0x5573921753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 -- 172.21.15.67:0/1057989370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557392175ad0 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297d400640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f297ca00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2968008f70 tx=0x7f29680045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c8a19c8cb319e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.643+0000 7f2966000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968033070 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9519+0+0 (secure 0 0 0) 0x7f296801d6f0 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2968003d70 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f296803f050 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 10 client.16524 fetch_fsmap learned FSMap version 37 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557392176df0 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2968003d70 con 0x5573920ecff0 2024-09-10T23:24:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f296801c410 con 0x5573920ecff0 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 10 client.16524 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 10 client.16524 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 20 client.16524 populate_metadata read hostname 'smithi067' 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921774e0 0x557392179900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 4 client.16524 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d31018d", "release": true, "await": true}] 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.644+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d31018d", "release": true, "await": true}) -- 0x557392061c90 con 0x5573921774e0 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.645+0000 7f296fe00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921774e0 0x557392179900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:24:04.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.645+0000 7f296fe00640 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921774e0 0x557392179900 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0c79d6c9f9aa2 server_cookie=98424bc06fefcede in_seq=0 out_seq=0 2024-09-10T23:24:04.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.645+0000 7f2966000640 10 client.16524 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:24:04.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.863+0000 7f2966000640 1 -- 172.21.15.67:0/1057989370 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557392061c90 con 0x5573921774e0 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.863+0000 7f2966000640 10 client.16524 handle_command_reply: tid=0 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.863+0000 7f2981fde740 1 client.16524 shutdown 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 msgr2=0x5573920fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2968008f70 tx=0x7f29680045f0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 shutdown_connections 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5573921774e0 0x557392179900 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573920f98a0 0x5573921753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573920ed910 0x5573920fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 --2- 172.21.15.67:0/1057989370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5573920ecff0 0x5573920fdfd0 secure :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2968008f70 tx=0x7f29680045f0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 >> 172.21.15.67:0/1057989370 conn(0x5573920eae20 msgr2=0x5573921047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.864+0000 7f2966000640 0 client.16524 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:24:04.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.865+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 shutdown_connections 2024-09-10T23:24:04.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.869+0000 7f2981fde740 1 -- 172.21.15.67:0/1057989370 wait complete. 2024-09-10T23:24:04.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.869+0000 7f2981fde740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T23:24:04.871 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:24:04.871 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 37, 2024-09-10T23:24:04.871 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:24:04.871 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 76, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "1d31018d": { 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 76, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 138.8, 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:24:04.872 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:24:04.873 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.871+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 msgr2=0x7f295c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.871+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 0x7f295c0b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5573920f0170 tx=0x7f29680084c0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.871+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 msgr2=0x7f297819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.871+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f296002a930 tx=0x7f2960003b90 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 shutdown_connections 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f295c0b5450 0x7f295c0b7910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f297810d120 0x7f29781a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29781045e0 0x7f297819b900 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 --2- 172.21.15.67:0/4147660734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2978103cc0 0x7f297819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:24:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.872+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 >> 172.21.15.67:0/4147660734 conn(0x7f29780fd700 msgr2=0x7f29780ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:24:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.873+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 shutdown_connections 2024-09-10T23:24:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:24:04.873+0000 7f297f400640 1 -- 172.21.15.67:0/4147660734 wait complete. 2024-09-10T23:24:06.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:05 smithi092 ceph-mon[34223]: pgmap v1179: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:06.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:05 smithi067 ceph-mon[30939]: pgmap v1179: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:06.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:05 smithi145 ceph-mon[34397]: pgmap v1179: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:06.982 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d31018d', total seconds elapsed: 28.1 2024-09-10T23:24:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:07 smithi092 ceph-mon[34223]: pgmap v1180: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-09-10T23:24:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:07 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:07 smithi067 ceph-mon[30939]: pgmap v1180: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-09-10T23:24:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:07 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:08.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:07 smithi145 ceph-mon[34397]: pgmap v1180: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-09-10T23:24:08.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:07 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:08 smithi092 ceph-mon[34223]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T23:24:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:08 smithi067 ceph-mon[30939]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T23:24:09.381 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:08 smithi145 ceph-mon[34397]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T23:24:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:09 smithi092 ceph-mon[34223]: pgmap v1182: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-10T23:24:10.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:09 smithi067 ceph-mon[30939]: pgmap v1182: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-10T23:24:10.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:09 smithi145 ceph-mon[34397]: pgmap v1182: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.9 MiB/s wr, 1.51k op/s 2024-09-10T23:24:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:11 smithi092 ceph-mon[34223]: pgmap v1183: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 1.54k op/s 2024-09-10T23:24:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:11 smithi067 ceph-mon[30939]: pgmap v1183: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 1.54k op/s 2024-09-10T23:24:12.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:11 smithi145 ceph-mon[34397]: pgmap v1183: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 1.54k op/s 2024-09-10T23:24:14.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:13 smithi092 ceph-mon[34223]: pgmap v1184: 129 pgs: 16 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 1.23k op/s 2024-09-10T23:24:14.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:13 smithi067 ceph-mon[30939]: pgmap v1184: 129 pgs: 16 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 1.23k op/s 2024-09-10T23:24:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:13 smithi145 ceph-mon[34397]: pgmap v1184: 129 pgs: 16 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 1.23k op/s 2024-09-10T23:24:16.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:15 smithi092 ceph-mon[34223]: pgmap v1185: 129 pgs: 20 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 64 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 1.84k op/s 2024-09-10T23:24:16.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:15 smithi067 ceph-mon[30939]: pgmap v1185: 129 pgs: 20 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 64 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 1.84k op/s 2024-09-10T23:24:16.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:15 smithi145 ceph-mon[34397]: pgmap v1185: 129 pgs: 20 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 64 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 1.84k op/s 2024-09-10T23:24:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:17 smithi092 ceph-mon[34223]: pgmap v1186: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 2.40k op/s 2024-09-10T23:24:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:17 smithi067 ceph-mon[30939]: pgmap v1186: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 2.40k op/s 2024-09-10T23:24:18.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:17 smithi145 ceph-mon[34397]: pgmap v1186: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 2.40k op/s 2024-09-10T23:24:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:20 smithi092 ceph-mon[34223]: pgmap v1187: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.64k op/s 2024-09-10T23:24:20.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:20 smithi067 ceph-mon[30939]: pgmap v1187: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.64k op/s 2024-09-10T23:24:20.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:20 smithi145 ceph-mon[34397]: pgmap v1187: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.64k op/s 2024-09-10T23:24:22.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:22 smithi092 ceph-mon[34223]: pgmap v1188: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-10T23:24:22.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:22 smithi067 ceph-mon[30939]: pgmap v1188: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-10T23:24:22.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:22 smithi145 ceph-mon[34397]: pgmap v1188: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-10T23:24:23.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:23 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:23 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:23.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:23 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:24:24.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:24 smithi145 ceph-mon[34397]: pgmap v1189: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:24:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:24.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:24 smithi092 ceph-mon[34223]: pgmap v1189: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:24:24.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:24.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:24 smithi067 ceph-mon[30939]: pgmap v1189: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:24:24.549 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:26.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:26 smithi145 ceph-mon[34397]: pgmap v1190: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:24:26.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:26 smithi092 ceph-mon[34223]: pgmap v1190: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:24:26.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:26 smithi067 ceph-mon[30939]: pgmap v1190: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T23:24:28.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:28 smithi145 ceph-mon[34397]: pgmap v1191: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:28.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:28 smithi092 ceph-mon[34223]: pgmap v1191: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:28.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:28 smithi067 ceph-mon[30939]: pgmap v1191: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:30.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:30 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:24:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:30 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:24:30.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:30 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:24:30.452 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:30 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:24:30.452 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:30 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:24:30.452 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:30 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:24:30.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:30 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:24:30.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:30 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:24:30.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:30 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:24:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:33 smithi067 ceph-mon[30939]: pgmap v1193: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:24:34.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:33 smithi067 ceph-mon[30939]: pgmap v1194: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-09-10T23:24:34.101 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:33 smithi145 ceph-mon[34397]: pgmap v1193: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:24:34.101 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:34 smithi145 ceph-mon[34397]: pgmap v1194: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-09-10T23:24:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:33 smithi092 ceph-mon[34223]: pgmap v1193: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:24:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:33 smithi092 ceph-mon[34223]: pgmap v1194: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-09-10T23:24:35.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:34 smithi145 ceph-mon[34397]: pgmap v1195: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-10T23:24:35.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:34 smithi092 ceph-mon[34223]: pgmap v1195: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-10T23:24:35.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:34 smithi067 ceph-mon[30939]: pgmap v1195: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 12 MiB/s wr, 3.00k op/s 2024-09-10T23:24:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:37 smithi092 ceph-mon[34223]: pgmap v1196: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:38.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:37 smithi067 ceph-mon[30939]: pgmap v1196: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:37 smithi145 ceph-mon[34397]: pgmap v1196: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-09-10T23:24:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:39 smithi092 ceph-mon[34223]: pgmap v1197: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:24:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:39 smithi067 ceph-mon[30939]: pgmap v1197: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:24:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:39 smithi145 ceph-mon[34397]: pgmap v1197: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:24:42.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:41 smithi092 ceph-mon[34223]: pgmap v1198: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-09-10T23:24:42.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:41 smithi067 ceph-mon[30939]: pgmap v1198: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-09-10T23:24:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:41 smithi145 ceph-mon[34397]: pgmap v1198: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-09-10T23:24:44.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:43 smithi092 ceph-mon[34223]: pgmap v1199: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T23:24:44.033 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:43 smithi067 ceph-mon[30939]: pgmap v1199: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T23:24:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:43 smithi145 ceph-mon[34397]: pgmap v1199: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T23:24:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:45 smithi092 ceph-mon[34223]: pgmap v1200: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:24:46.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:45 smithi067 ceph-mon[30939]: pgmap v1200: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:24:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:45 smithi145 ceph-mon[34397]: pgmap v1200: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T23:24:48.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:47 smithi092 ceph-mon[34223]: pgmap v1201: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:24:48.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:47 smithi067 ceph-mon[30939]: pgmap v1201: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:24:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:47 smithi145 ceph-mon[34397]: pgmap v1201: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:24:50.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:49 smithi092 ceph-mon[34223]: pgmap v1202: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T23:24:50.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:49 smithi067 ceph-mon[30939]: pgmap v1202: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T23:24:50.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:49 smithi145 ceph-mon[34397]: pgmap v1202: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T23:24:52.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:51 smithi092 ceph-mon[34223]: pgmap v1203: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:52.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:51 smithi067 ceph-mon[30939]: pgmap v1203: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:51 smithi145 ceph-mon[34397]: pgmap v1203: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 3.21k op/s 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: pgmap v1204: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:53 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: pgmap v1204: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:53 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: pgmap v1204: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:24:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:53 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:24:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:55 smithi092 ceph-mon[34223]: pgmap v1205: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-09-10T23:24:56.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:55 smithi067 ceph-mon[30939]: pgmap v1205: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-09-10T23:24:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:55 smithi145 ceph-mon[34397]: pgmap v1205: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-09-10T23:24:57.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:56 smithi067 ceph-mon[30939]: pgmap v1206: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:24:57.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:56 smithi145 ceph-mon[34397]: pgmap v1206: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:24:57.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:56 smithi092 ceph-mon[34223]: pgmap v1206: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 2.98k op/s 2024-09-10T23:25:00.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:24:59 smithi092 ceph-mon[34223]: pgmap v1207: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:00.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:24:59 smithi067 ceph-mon[30939]: pgmap v1207: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:24:59 smithi145 ceph-mon[34397]: pgmap v1207: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:01.981 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:01 smithi092 ceph-mon[34223]: pgmap v1208: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:25:02.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:01 smithi067 ceph-mon[30939]: pgmap v1208: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:25:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:01 smithi145 ceph-mon[34397]: pgmap v1208: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: pgmap v1209: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:25:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:03 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: pgmap v1209: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:04.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:25:04.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:25:04.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:03 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: pgmap v1209: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:04.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:04.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:25:04.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:25:04.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:03 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:25:06.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:05 smithi092 ceph-mon[34223]: pgmap v1210: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-10T23:25:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:05 smithi067 ceph-mon[30939]: pgmap v1210: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-10T23:25:06.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:05 smithi145 ceph-mon[34397]: pgmap v1210: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-10T23:25:08.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:07 smithi092 ceph-mon[34223]: pgmap v1211: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T23:25:08.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:07 smithi067 ceph-mon[30939]: pgmap v1211: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T23:25:08.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:07 smithi145 ceph-mon[34397]: pgmap v1211: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T23:25:09.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:08 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:09.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:08 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:09.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:08 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:10.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:09 smithi092 ceph-mon[34223]: pgmap v1212: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-10T23:25:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:09 smithi067 ceph-mon[30939]: pgmap v1212: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-10T23:25:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:09 smithi145 ceph-mon[34397]: pgmap v1212: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-10T23:25:12.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:11 smithi092 ceph-mon[34223]: pgmap v1213: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T23:25:12.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:11 smithi067 ceph-mon[30939]: pgmap v1213: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T23:25:12.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:11 smithi145 ceph-mon[34397]: pgmap v1213: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T23:25:14.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:13 smithi092 ceph-mon[34223]: pgmap v1214: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:25:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:13 smithi067 ceph-mon[30939]: pgmap v1214: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:25:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:13 smithi145 ceph-mon[34397]: pgmap v1214: 129 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T23:25:15.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:14 smithi067 ceph-mon[30939]: pgmap v1215: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-09-10T23:25:15.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:14 smithi145 ceph-mon[34397]: pgmap v1215: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-09-10T23:25:15.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:14 smithi092 ceph-mon[34223]: pgmap v1215: 129 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 113 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-09-10T23:25:16.997 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:25:16.998 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:25:16.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:25:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:17 smithi092 ceph-mon[34223]: pgmap v1216: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T23:25:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:17 smithi067 ceph-mon[30939]: pgmap v1216: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T23:25:18.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:17 smithi145 ceph-mon[34397]: pgmap v1216: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T23:25:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:19 smithi092 ceph-mon[34223]: pgmap v1217: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-09-10T23:25:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:19 smithi092 ceph-mon[34223]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T23:25:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:19 smithi067 ceph-mon[30939]: pgmap v1217: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-09-10T23:25:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:19 smithi067 ceph-mon[30939]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T23:25:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:19 smithi145 ceph-mon[34397]: pgmap v1217: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-09-10T23:25:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:19 smithi145 ceph-mon[34397]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T23:25:22.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:21 smithi092 ceph-mon[34223]: pgmap v1219: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 2.52k op/s 2024-09-10T23:25:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:21 smithi067 ceph-mon[30939]: pgmap v1219: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 2.52k op/s 2024-09-10T23:25:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:21 smithi145 ceph-mon[34397]: pgmap v1219: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 2.52k op/s 2024-09-10T23:25:23.984 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:25:24.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:23 smithi092 ceph-mon[34223]: pgmap v1220: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-09-10T23:25:24.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:23 smithi067 ceph-mon[30939]: pgmap v1220: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-09-10T23:25:24.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:23 smithi145 ceph-mon[34397]: pgmap v1220: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-09-10T23:25:26.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:25 smithi092 ceph-mon[34223]: pgmap v1221: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.8 MiB/s wr, 2.47k op/s 2024-09-10T23:25:26.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:25 smithi067 ceph-mon[30939]: pgmap v1221: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.8 MiB/s wr, 2.47k op/s 2024-09-10T23:25:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:25 smithi145 ceph-mon[34397]: pgmap v1221: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.8 MiB/s wr, 2.47k op/s 2024-09-10T23:25:28.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:27 smithi092 ceph-mon[34223]: pgmap v1222: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-09-10T23:25:28.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:27 smithi067 ceph-mon[30939]: pgmap v1222: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-09-10T23:25:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:27 smithi145 ceph-mon[34397]: pgmap v1222: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-09-10T23:25:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.206+0000 7f9f51600640 1 -- 172.21.15.67:0/2096161665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 msgr2=0x7f9f4c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.206+0000 7f9f51600640 1 --2- 172.21.15.67:0/2096161665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1053a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009be0 tx=0x7f9f4402a6e0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.208+0000 7f9f51600640 1 -- 172.21.15.67:0/2096161665 shutdown_connections 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.208+0000 7f9f51600640 1 --2- 172.21.15.67:0/2096161665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10b350 0x7f9f4c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.208+0000 7f9f51600640 1 --2- 172.21.15.67:0/2096161665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.208+0000 7f9f51600640 1 --2- 172.21.15.67:0/2096161665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1053a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.208+0000 7f9f51600640 1 -- 172.21.15.67:0/2096161665 >> 172.21.15.67:0/2096161665 conn(0x7f9f4c1009f0 msgr2=0x7f9f4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.209+0000 7f9f51600640 1 -- 172.21.15.67:0/2096161665 shutdown_connections 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.209+0000 7f9f51600640 1 -- 172.21.15.67:0/2096161665 wait complete. 2024-09-10T23:25:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.212+0000 7f9f51600640 1 Processor -- start 2024-09-10T23:25:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.213+0000 7f9f51600640 1 -- start start 2024-09-10T23:25:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.215+0000 7f9f51600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.215+0000 7f9f51600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.215+0000 7f9f51600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 0x7f9f4c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.216+0000 7f9f4a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.216+0000 7f9f4a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43580/0 (socket says 172.21.15.67:43580) 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.218+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c079c80 con 0x7f9f4c10a9d0 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.218+0000 7f9f4a000640 1 -- 172.21.15.67:0/4173728458 learned_addr learned my addr 172.21.15.67:0/4173728458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.218+0000 7f9f4b400640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 0x7f9f4c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.218+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c079b00 con 0x7f9f4c10b350 2024-09-10T23:25:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.218+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c079e00 con 0x7f9f4c104fc0 2024-09-10T23:25:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4aa00640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4b400640 1 -- 172.21.15.67:0/4173728458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 msgr2=0x7f9f4c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4b400640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4b400640 1 -- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 msgr2=0x7f9f4c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4b400640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.219+0000 7f9f4b400640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c19d0b0 con 0x7f9f4c10b350 2024-09-10T23:25:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.225+0000 7f9f4a000640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:25:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.225+0000 7f9f4b400640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 0x7f9f4c19c8e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c710 tx=0x7f9f3c00cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.226+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004220 con 0x7f9f4c10b350 2024-09-10T23:25:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.226+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f3c007e40 con 0x7f9f4c10b350 2024-09-10T23:25:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.226+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c003d40 con 0x7f9f4c10b350 2024-09-10T23:25:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.226+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c19d3a0 con 0x7f9f4c10b350 2024-09-10T23:25:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.226+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1a3a60 con 0x7f9f4c10b350 2024-09-10T23:25:29.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.228+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c1053a0 con 0x7f9f4c10b350 2024-09-10T23:25:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.233+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9f3c047b70 con 0x7f9f4c10b350 2024-09-10T23:25:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.236+0000 7f9f43400640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 0x7f9f300b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.236+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f9f3c0e62e0 con 0x7f9f4c10b350 2024-09-10T23:25:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.237+0000 7f9f4aa00640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 0x7f9f300b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.239+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9f3c014030 con 0x7f9f4c10b350 2024-09-10T23:25:29.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.242+0000 7f9f4aa00640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 0x7f9f300b7a60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.713+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f4c1a3cf0 con 0x7f9f4c10b350 2024-09-10T23:25:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.714+0000 7f9f43400640 1 -- 172.21.15.67:0/4173728458 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11904 (secure 0 0 0) 0x7f9f3c0acae0 con 0x7f9f4c10b350 2024-09-10T23:25:29.719 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:25:29.719 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":40,"btime":"2024-09-10T23:25:07:702108+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:25:07.260218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T23:25:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 40 2024-09-10T23:25:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 msgr2=0x7f9f300b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 0x7f9f300b7a60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44000f40 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 msgr2=0x7f9f4c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 0x7f9f4c19c8e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c710 tx=0x7f9f3c00cbd0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 shutdown_connections 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9f300b55a0 0x7f9f300b7a60 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44000f40 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f4c10b350 0x7f9f4c19c8e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f4c10a9d0 0x7f9f4c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 --2- 172.21.15.67:0/4173728458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104fc0 0x7f9f4c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.718+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 >> 172.21.15.67:0/4173728458 conn(0x7f9f4c1009f0 msgr2=0x7f9f4c1022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.719+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 shutdown_connections 2024-09-10T23:25:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:29.719+0000 7f9f51600640 1 -- 172.21.15.67:0/4173728458 wait complete. 2024-09-10T23:25:30.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:29 smithi092 ceph-mon[34223]: pgmap v1223: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:29 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4173728458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:29 smithi067 ceph-mon[30939]: pgmap v1223: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:30.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:29 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4173728458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:30.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:29 smithi145 ceph-mon[34397]: pgmap v1223: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:29 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4173728458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:31 smithi067 ceph-mon[30939]: pgmap v1224: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T23:25:32.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:31 smithi145 ceph-mon[34397]: pgmap v1224: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T23:25:32.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:31 smithi092 ceph-mon[34223]: pgmap v1224: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T23:25:32.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:32 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:33.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:32 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:33.167 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:32 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:25:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:33 smithi092 ceph-mon[34223]: pgmap v1225: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:25:34.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:33 smithi067 ceph-mon[30939]: pgmap v1225: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:25:34.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:33 smithi145 ceph-mon[34397]: pgmap v1225: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T23:25:35.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:34 smithi092 ceph-mon[34223]: pgmap v1226: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:35.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:34 smithi067 ceph-mon[30939]: pgmap v1226: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:35.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:34 smithi145 ceph-mon[34397]: pgmap v1226: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:36.265 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:25:36.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:25:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:37 smithi092 ceph-mon[34223]: pgmap v1227: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:38.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:37 smithi067 ceph-mon[30939]: pgmap v1227: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:37 smithi145 ceph-mon[34397]: pgmap v1227: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T23:25:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:25:40.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:39 smithi092 ceph-mon[34223]: pgmap v1228: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-09-10T23:25:40.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:39 smithi067 ceph-mon[30939]: pgmap v1228: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-09-10T23:25:40.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:39 smithi145 ceph-mon[34397]: pgmap v1228: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-09-10T23:25:42.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:41 smithi092 ceph-mon[34223]: pgmap v1229: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:42.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:41 smithi067 ceph-mon[30939]: pgmap v1229: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:41 smithi145 ceph-mon[34397]: pgmap v1229: 129 pgs: 16 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:43.747 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:43 smithi067 ceph-mon[30939]: pgmap v1230: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:25:43.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:43 smithi092 ceph-mon[34223]: pgmap v1230: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:25:44.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:43 smithi145 ceph-mon[34397]: pgmap v1230: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:25:44.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.892+0000 7f2652200640 1 -- 172.21.15.67:0/911102179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c10a1f0 msgr2=0x7f264c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:44.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.892+0000 7f2652200640 1 --2- 172.21.15.67:0/911102179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c10a1f0 0x7f264c10a5d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f263c009900 tx=0x7f263c02a4a0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 -- 172.21.15.67:0/911102179 shutdown_connections 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 --2- 172.21.15.67:0/911102179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c113710 0x7f264c115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 --2- 172.21.15.67:0/911102179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 --2- 172.21.15.67:0/911102179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c10a1f0 0x7f264c10a5d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 -- 172.21.15.67:0/911102179 >> 172.21.15.67:0/911102179 conn(0x7f264c0788a0 msgr2=0x7f264c07acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 -- 172.21.15.67:0/911102179 shutdown_connections 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.893+0000 7f2652200640 1 -- 172.21.15.67:0/911102179 wait complete. 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.895+0000 7f2652200640 1 Processor -- start 2024-09-10T23:25:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.896+0000 7f2652200640 1 -- start start 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 0x7f264c074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1193a0 con 0x7f264c10ab10 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f264c119220 con 0x7f264c113710 2024-09-10T23:25:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.897+0000 7f2652200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f264c119520 con 0x7f264c10a1f0 2024-09-10T23:25:44.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.898+0000 7f264be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.898+0000 7f264be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:53964/0 (socket says 172.21.15.67:53964) 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.898+0000 7f264be00640 1 -- 172.21.15.67:0/2682370011 learned_addr learned my addr 172.21.15.67:0/2682370011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.898+0000 7f264aa00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.899+0000 7f264aa00640 1 -- 172.21.15.67:0/2682370011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 msgr2=0x7f264c074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.898+0000 7f264b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 0x7f264c074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.901+0000 7f264aa00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 0x7f264c074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.901+0000 7f264aa00640 1 -- 172.21.15.67:0/2682370011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 msgr2=0x7f264c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:44.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.902+0000 7f264aa00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:44.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.902+0000 7f264aa00640 1 -- 172.21.15.67:0/2682370011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c06e4c0 con 0x7f264c10ab10 2024-09-10T23:25:44.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.902+0000 7f264b400640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 0x7f264c074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:25:44.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.904+0000 7f264be00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:25:44.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.907+0000 7f264aa00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c074d50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f263800b3c0 tx=0x7f263800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:44.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.908+0000 7f2643e00640 1 -- 172.21.15.67:0/2682370011 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638011830 con 0x7f264c10ab10 2024-09-10T23:25:44.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.908+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c06e7b0 con 0x7f264c10ab10 2024-09-10T23:25:44.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.908+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c073810 con 0x7f264c10ab10 2024-09-10T23:25:44.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.909+0000 7f2643e00640 1 -- 172.21.15.67:0/2682370011 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2638016070 con 0x7f264c10ab10 2024-09-10T23:25:44.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.909+0000 7f2643e00640 1 -- 172.21.15.67:0/2682370011 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263801bce0 con 0x7f264c10ab10 2024-09-10T23:25:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.918+0000 7f2643e00640 1 -- 172.21.15.67:0/2682370011 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2638010380 con 0x7f264c10ab10 2024-09-10T23:25:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.920+0000 7f2643e00640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 0x7f26300b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.921+0000 7f2643e00640 1 -- 172.21.15.67:0/2682370011 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f26380e6920 con 0x7f264c10ab10 2024-09-10T23:25:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:44.922+0000 7f264b400640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 0x7f26300b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.226+0000 7f264b400640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 0x7f26300b7a10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f263c006010 tx=0x7f263c000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.228+0000 7f2654caa740 1 Processor -- start 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.229+0000 7f2654caa740 1 -- start start 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.230+0000 7f2654caa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 0x55787229b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.231+0000 7f2654caa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.231+0000 7f2654caa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.231+0000 7f2654caa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557872210ca0 con 0x5578722a7a30 2024-09-10T23:25:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.232+0000 7f2654caa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578722000d0 con 0x55787229baa0 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.232+0000 7f2654caa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557872203ee0 con 0x55787229b180 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.232+0000 7f264b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 0x55787229b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.232+0000 7f264aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.233+0000 7f264aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:53980/0 (socket says 172.21.15.67:53980) 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.233+0000 7f264aa00640 1 -- 172.21.15.67:0/1003690278 learned_addr learned my addr 172.21.15.67:0/1003690278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:45.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.232+0000 7f264b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 0x55787229b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:38660/0 (socket says 172.21.15.67:38660) 2024-09-10T23:25:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.234+0000 7f264be00640 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.235+0000 7f264aa00640 1 -- 172.21.15.67:0/1003690278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 msgr2=0x55787229b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.235+0000 7f264aa00640 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 0x55787229b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.235+0000 7f264aa00640 1 -- 172.21.15.67:0/1003690278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 msgr2=0x5578722a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.235+0000 7f264aa00640 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.235+0000 7f264aa00640 1 -- 172.21.15.67:0/1003690278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578721b2400 con 0x55787229baa0 2024-09-10T23:25:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.237+0000 7f264aa00640 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f263800bfd0 tx=0x7f26380092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd91e126f29f102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.237+0000 7f2637e00640 1 -- 172.21.15.67:0/1003690278 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380b1170 con 0x55787229baa0 2024-09-10T23:25:45.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.237+0000 7f2637e00640 1 -- 172.21.15.67:0/1003690278 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26380acfc0 con 0x55787229baa0 2024-09-10T23:25:45.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.237+0000 7f2637e00640 1 -- 172.21.15.67:0/1003690278 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380e63e0 con 0x55787229baa0 2024-09-10T23:25:45.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 -- 172.21.15.67:0/1003690278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 msgr2=0x5578722a74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:45.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f263800bfd0 tx=0x7f26380092c0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 -- 172.21.15.67:0/1003690278 shutdown_connections 2024-09-10T23:25:45.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a74f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 --2- 172.21.15.67:0/1003690278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55787229b180 0x55787229b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.239+0000 7f2654caa740 1 -- 172.21.15.67:0/1003690278 >> 172.21.15.67:0/1003690278 conn(0x557872298fb0 msgr2=0x55787229a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:45.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.240+0000 7f2654caa740 1 -- 172.21.15.67:0/1003690278 shutdown_connections 2024-09-10T23:25:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.240+0000 7f2654caa740 1 -- 172.21.15.67:0/1003690278 wait complete. 2024-09-10T23:25:45.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.241+0000 7f2654caa740 1 -- start start 2024-09-10T23:25:45.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.242+0000 7f2654caa740 1 Processor -- start 2024-09-10T23:25:45.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55787229b180 0x5578722a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5578720617a0 con 0x55787229b180 2024-09-10T23:25:45.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578721a5a10 con 0x55787229baa0 2024-09-10T23:25:45.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5578722aa6b0 con 0x5578722a7a30 2024-09-10T23:25:45.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.245+0000 7f2654caa740 10 client.0 _ll_register_callbacks cb 0x5578721b5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:25:45.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.249+0000 7f264be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.249+0000 7f264be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:38662/0 (socket says 172.21.15.67:38662) 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.249+0000 7f264be00640 1 -- 172.21.15.67:0/770045844 learned_addr learned my addr 172.21.15.67:0/770045844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264b400640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55787229b180 0x5578722a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 -- 172.21.15.67:0/770045844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 msgr2=0x5578722a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 -- 172.21.15.67:0/770045844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55787229b180 msgr2=0x5578722a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55787229b180 0x5578722a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.250+0000 7f264aa00640 1 -- 172.21.15.67:0/770045844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557871f6db90 con 0x55787229baa0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.251+0000 7f264aa00640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a47c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x557872299e90 tx=0x7f26380a8d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e54497345f571c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.251+0000 7f2634c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.251+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380d3020 con 0x55787229baa0 2024-09-10T23:25:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.251+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f26380abd20 con 0x55787229baa0 2024-09-10T23:25:45.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.251+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26380acfc0 con 0x55787229baa0 2024-09-10T23:25:45.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.252+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f26380b1170 con 0x55787229baa0 2024-09-10T23:25:45.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.252+0000 7f2654caa740 10 client.36260 fetch_fsmap learned FSMap version 41 2024-09-10T23:25:45.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.252+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5578722b60c0 con 0x55787229baa0 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.252+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26380acfc0 con 0x55787229baa0 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.253+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f26380ab590 con 0x55787229baa0 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.253+0000 7f2654caa740 10 client.36260 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.253+0000 7f2654caa740 10 client.36260 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.253+0000 7f2654caa740 20 client.36260 populate_metadata read hostname 'smithi067' 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.253+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5578722b7410 0x5578722b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.254+0000 7f2654caa740 4 client.36260 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:25:45.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.254+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557872212bf0 con 0x5578722b7410 2024-09-10T23:25:45.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.254+0000 7f264b400640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5578722b7410 0x5578722b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.256+0000 7f264b400640 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5578722b7410 0x5578722b9800 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ccc1e27c6709559 server_cookie=37421e59cea58fff in_seq=0 out_seq=0 2024-09-10T23:25:45.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.256+0000 7f2634c00640 10 client.36260 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:25:45.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.257+0000 7f2634c00640 1 -- 172.21.15.67:0/770045844 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557872212bf0 con 0x5578722b7410 2024-09-10T23:25:45.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.257+0000 7f2634c00640 10 client.36260 handle_command_reply: tid=0 2024-09-10T23:25:45.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.259+0000 7f2654caa740 1 client.36260 shutdown 2024-09-10T23:25:45.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.259+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 msgr2=0x5578722a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:45.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.259+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a47c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x557872299e90 tx=0x7f26380a8d60 comp rx=0 tx=0).stop 2024-09-10T23:25:45.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 shutdown_connections 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5578722b7410 0x5578722b9800 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229baa0 0x5578722a47c0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x557872299e90 tx=0x7f26380a8d60 comp rx=0 tx=0).stop 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 --2- 172.21.15.67:0/770045844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55787229b180 0x5578722a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 >> 172.21.15.67:0/770045844 conn(0x557872298fb0 msgr2=0x55787229eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.260+0000 7f2634c00640 0 client.36260 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.262+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 shutdown_connections 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.262+0000 7f2654caa740 1 -- 172.21.15.67:0/770045844 wait complete. 2024-09-10T23:25:45.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.262+0000 7f2654caa740 20 client.36260 trim_cache size 0 max 16384 2024-09-10T23:25:45.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 Processor -- start 2024-09-10T23:25:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 -- start start 2024-09-10T23:25:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a3580 0x5578722a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f262c012ad0 con 0x5578722a7a30 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f262c010340 con 0x55787229b180 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.303+0000 7f2654caa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00e1b0 con 0x5578722a3580 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.304+0000 7f264b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.304+0000 7f264b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:54000/0 (socket says 172.21.15.67:54000) 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.304+0000 7f264b400640 1 -- 172.21.15.67:0/697429762 learned_addr learned my addr 172.21.15.67:0/697429762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.306+0000 7f264be00640 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a3580 0x5578722a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.306+0000 7f264b400640 1 -- 172.21.15.67:0/697429762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a3580 msgr2=0x5578722a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.306+0000 7f264b400640 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a3580 0x5578722a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.307+0000 7f264b400640 1 -- 172.21.15.67:0/697429762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 msgr2=0x5578722a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.307+0000 7f264b400640 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.307+0000 7f264b400640 1 -- 172.21.15.67:0/697429762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557872298de0 con 0x55787229b180 2024-09-10T23:25:45.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.307+0000 7f264b400640 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f263c033000 tx=0x7f263c03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96269858a71fb8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.308+0000 7f2637400640 1 -- 172.21.15.67:0/697429762 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c035050 con 0x55787229b180 2024-09-10T23:25:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.308+0000 7f2637400640 1 -- 172.21.15.67:0/697429762 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f263c041070 con 0x55787229b180 2024-09-10T23:25:45.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.308+0000 7f2637400640 1 -- 172.21.15.67:0/697429762 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c04a760 con 0x55787229b180 2024-09-10T23:25:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 -- 172.21.15.67:0/697429762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 msgr2=0x5578722988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f263c033000 tx=0x7f263c03e000 comp rx=0 tx=0).stop 2024-09-10T23:25:45.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 -- 172.21.15.67:0/697429762 shutdown_connections 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a3580 0x5578722a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a7a30 0x5578722a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 --2- 172.21.15.67:0/697429762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722988a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.310+0000 7f2654caa740 1 -- 172.21.15.67:0/697429762 >> 172.21.15.67:0/697429762 conn(0x557872298fb0 msgr2=0x557872299ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.311+0000 7f2654caa740 1 -- 172.21.15.67:0/697429762 shutdown_connections 2024-09-10T23:25:45.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.311+0000 7f2654caa740 1 -- 172.21.15.67:0/697429762 wait complete. 2024-09-10T23:25:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.312+0000 7f2654caa740 1 -- start start 2024-09-10T23:25:45.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.312+0000 7f2654caa740 1 Processor -- start 2024-09-10T23:25:45.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.314+0000 7f2654caa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a3580 0x5578722b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00e800 con 0x5578722a3580 2024-09-10T23:25:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00fc70 con 0x55787229b180 2024-09-10T23:25:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00eed0 con 0x5578722a7a30 2024-09-10T23:25:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.315+0000 7f2654caa740 10 client.0 _ll_register_callbacks cb 0x7f26380b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:25:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.316+0000 7f264b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.316+0000 7f264b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:54004/0 (socket says 172.21.15.67:54004) 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.316+0000 7f264b400640 1 -- 172.21.15.67:0/3153283646 learned_addr learned my addr 172.21.15.67:0/3153283646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.317+0000 7f264b400640 1 -- 172.21.15.67:0/3153283646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 msgr2=0x5578722b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.317+0000 7f264b400640 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.317+0000 7f264b400640 1 -- 172.21.15.67:0/3153283646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a3580 msgr2=0x5578722b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.317+0000 7f264b400640 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a3580 0x5578722b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.317+0000 7f264b400640 1 -- 172.21.15.67:0/3153283646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578722b9960 con 0x55787229b180 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.318+0000 7f264b400640 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f263c005d00 tx=0x7f263c03b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eeddba648286511 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.318+0000 7f2634c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:25:45.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.319+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c035030 con 0x55787229b180 2024-09-10T23:25:45.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.319+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f263c03bc10 con 0x55787229b180 2024-09-10T23:25:45.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.320+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263c041070 con 0x55787229b180 2024-09-10T23:25:45.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.320+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f263c03be10 con 0x55787229b180 2024-09-10T23:25:45.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.320+0000 7f2654caa740 10 client.36266 fetch_fsmap learned FSMap version 41 2024-09-10T23:25:45.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.320+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557872324d50 con 0x55787229b180 2024-09-10T23:25:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f263c041070 con 0x55787229b180 2024-09-10T23:25:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f263c04a5a0 con 0x55787229b180 2024-09-10T23:25:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 10 client.36266 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:25:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 10 client.36266 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:25:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 20 client.36266 populate_metadata read hostname 'smithi067' 2024-09-10T23:25:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557872325410 0x557872327830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 4 client.36266 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:25:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.322+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55787220e6a0 con 0x557872325410 2024-09-10T23:25:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.323+0000 7f264aa00640 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557872325410 0x557872327830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:45.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.324+0000 7f2634c00640 10 client.36266 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:25:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:45.325+0000 7f264aa00640 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557872325410 0x557872327830 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd3930299646daf server_cookie=403ec10b2eedb6ad in_seq=0 out_seq=0 2024-09-10T23:25:46.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:45 smithi092 ceph-mon[34223]: pgmap v1231: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 2.35k op/s 2024-09-10T23:25:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:45 smithi067 ceph-mon[30939]: pgmap v1231: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 2.35k op/s 2024-09-10T23:25:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:45 smithi145 ceph-mon[34397]: pgmap v1231: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 2.35k op/s 2024-09-10T23:25:48.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:47 smithi092 ceph-mon[34223]: pgmap v1232: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-10T23:25:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:47 smithi067 ceph-mon[30939]: pgmap v1232: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-10T23:25:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:47 smithi145 ceph-mon[34397]: pgmap v1232: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-10T23:25:50.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:49 smithi092 ceph-mon[34223]: pgmap v1233: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:50.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:49 smithi067 ceph-mon[30939]: pgmap v1233: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:49 smithi145 ceph-mon[34397]: pgmap v1233: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-09-10T23:25:52.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:51 smithi092 ceph-mon[34223]: pgmap v1234: 129 pgs: 13 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 77 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:25:52.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:51 smithi067 ceph-mon[30939]: pgmap v1234: 129 pgs: 13 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 77 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:25:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:51 smithi145 ceph-mon[34397]: pgmap v1234: 129 pgs: 13 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 77 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-10T23:25:53.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.386+0000 7f2634c00640 1 -- 172.21.15.67:0/3153283646 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55787220e6a0 con 0x557872325410 2024-09-10T23:25:53.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.386+0000 7f2634c00640 10 client.36266 handle_command_reply: tid=0 2024-09-10T23:25:53.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.386+0000 7f2654caa740 1 client.36266 shutdown 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.386+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 msgr2=0x5578722bc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.386+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f263c005d00 tx=0x7f263c03b550 comp rx=0 tx=0).stop 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 shutdown_connections 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x557872325410 0x557872327830 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2634c00640 0 client.36266 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:25:53.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578722a7a30 0x5578722b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5578722a3580 0x5578722b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 --2- 172.21.15.67:0/3153283646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55787229b180 0x5578722bc980 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 >> 172.21.15.67:0/3153283646 conn(0x557872298fb0 msgr2=0x557872299ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 shutdown_connections 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.387+0000 7f2654caa740 1 -- 172.21.15.67:0/3153283646 wait complete. 2024-09-10T23:25:53.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.388+0000 7f2654caa740 20 client.36266 trim_cache size 0 max 16384 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 41, 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 78, 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "7fa737a0": { 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 78, 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:25:53.390 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:25:53.391 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:25:53.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:25:53.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:25:53.393 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:25:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 msgr2=0x7f26300b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 0x7f26300b7a10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f263c006010 tx=0x7f263c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 msgr2=0x7f264c074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c074d50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f263800b3c0 tx=0x7f263800b880 comp rx=0 tx=0).stop 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 shutdown_connections 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f26300b5550 0x7f26300b7a10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264c113710 0x7f264c06dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f264c10ab10 0x7f264c074d50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 --2- 172.21.15.67:0/2682370011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c10a1f0 0x7f264c074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.389+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 >> 172.21.15.67:0/2682370011 conn(0x7f264c0788a0 msgr2=0x7f264c079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.390+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 shutdown_connections 2024-09-10T23:25:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:53.390+0000 7f2652200640 1 -- 172.21.15.67:0/2682370011 wait complete. 2024-09-10T23:25:54.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:53 smithi067 ceph-mon[30939]: pgmap v1235: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:54.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:53 smithi145 ceph-mon[34397]: pgmap v1235: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:54.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:53 smithi092 ceph-mon[34223]: pgmap v1235: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T23:25:56.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:55 smithi067 ceph-mon[30939]: pgmap v1236: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:25:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:55 smithi145 ceph-mon[34397]: pgmap v1236: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:25:56.256 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7fa737a0', quiesce took 39.3 seconds. Will release after: -9.299999999999997 2024-09-10T23:25:56.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:25:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:55 smithi092 ceph-mon[34223]: pgmap v1236: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:25:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:25:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 -- 172.21.15.67:0/4208337730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f810f200 msgr2=0x7f39f81115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 --2- 172.21.15.67:0/4208337730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f810f200 0x7f39f81115f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009ca0 tx=0x7f39f002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 -- 172.21.15.67:0/4208337730 shutdown_connections 2024-09-10T23:25:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 --2- 172.21.15.67:0/4208337730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f810f200 0x7f39f81115f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 --2- 172.21.15.67:0/4208337730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f8075e00 0x7f39f810ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 --2- 172.21.15.67:0/4208337730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f80758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.402+0000 7f39fcc00640 1 -- 172.21.15.67:0/4208337730 >> 172.21.15.67:0/4208337730 conn(0x7f39f80fe660 msgr2=0x7f39f8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.403+0000 7f39fcc00640 1 -- 172.21.15.67:0/4208337730 shutdown_connections 2024-09-10T23:25:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.403+0000 7f39fcc00640 1 -- 172.21.15.67:0/4208337730 wait complete. 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 Processor -- start 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 -- start start 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 0x7f39f819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f810f200 0x7f39f81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8114ce0 con 0x7f39f8075e00 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8114b60 con 0x7f39f80754e0 2024-09-10T23:25:57.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39fcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8114e60 con 0x7f39f810f200 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39f7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39f7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:59392/0 (socket says 172.21.15.67:59392) 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.404+0000 7f39f7400640 1 -- 172.21.15.67:0/2298894583 learned_addr learned my addr 172.21.15.67:0/2298894583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7e00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f810f200 0x7f39f81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39efe00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 0x7f39f819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 -- 172.21.15.67:0/2298894583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f810f200 msgr2=0x7f39f81a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f810f200 0x7f39f81a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 -- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 msgr2=0x7f39f819b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 0x7f39f819b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 -- 172.21.15.67:0/2298894583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f81a1140 con 0x7f39f80754e0 2024-09-10T23:25:57.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39efe00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 0x7f39f819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.405+0000 7f39f7400640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c920 tx=0x7f39e800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.406+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8007c70 con 0x7f39f80754e0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.406+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39e8007e10 con 0x7f39f80754e0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.406+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8018710 con 0x7f39f80754e0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.406+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f81a1430 con 0x7f39f80754e0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.406+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f81a7af0 con 0x7f39f80754e0 2024-09-10T23:25:57.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.407+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f80758c0 con 0x7f39f80754e0 2024-09-10T23:25:57.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.409+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f39e800f450 con 0x7f39f80754e0 2024-09-10T23:25:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.411+0000 7f39f4c00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 0x7f39dc0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:25:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.411+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f39e80e67f0 con 0x7f39f80754e0 2024-09-10T23:25:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.411+0000 7f39efe00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 0x7f39dc0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:25:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.412+0000 7f39efe00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 0x7f39dc0b79c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39f8100d60 tx=0x7f39e0005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:25:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.412+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f39e8014030 con 0x7f39f80754e0 2024-09-10T23:25:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.655+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39f802e5c0 con 0x7f39f80754e0 2024-09-10T23:25:57.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.656+0000 7f39f4c00640 1 -- 172.21.15.67:0/2298894583 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11905 (secure 0 0 0) 0x7f39e80ace90 con 0x7f39f80754e0 2024-09-10T23:25:57.657 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:25:57.657 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"btime":"2024-09-10T23:25:31:756758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:25:31.060860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:25:57.657 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-09-10T23:25:57.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 msgr2=0x7f39dc0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:57.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 0x7f39dc0b79c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39f8100d60 tx=0x7f39e0005e50 comp rx=0 tx=0).stop 2024-09-10T23:25:57.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 msgr2=0x7f39f819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c920 tx=0x7f39e800cde0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 shutdown_connections 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f39dc0b5500 0x7f39dc0b79c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39f810f200 0x7f39f81a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39f8075e00 0x7f39f819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 --2- 172.21.15.67:0/2298894583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39f80754e0 0x7f39f819b090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 >> 172.21.15.67:0/2298894583 conn(0x7f39f80fe660 msgr2=0x7f39f8106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 shutdown_connections 2024-09-10T23:25:57.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:25:57.658+0000 7f39fcc00640 1 -- 172.21.15.67:0/2298894583 wait complete. 2024-09-10T23:25:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:57 smithi067 ceph-mon[30939]: pgmap v1237: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:57 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2298894583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:57 smithi145 ceph-mon[34397]: pgmap v1237: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:57 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2298894583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:57 smithi092 ceph-mon[34223]: pgmap v1237: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T23:25:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:57 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2298894583' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:25:59.113 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:58 smithi067 ceph-mon[30939]: pgmap v1238: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.9 MiB/s wr, 1.73k op/s 2024-09-10T23:25:59.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:58 smithi145 ceph-mon[34397]: pgmap v1238: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.9 MiB/s wr, 1.73k op/s 2024-09-10T23:25:59.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:58 smithi092 ceph-mon[34223]: pgmap v1238: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.9 MiB/s wr, 1.73k op/s 2024-09-10T23:25:59.333 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id 7fa737a0 --release --await' 2024-09-10T23:25:59.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id 7fa737a0 --release --await 2024-09-10T23:25:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:26:00.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:25:59 smithi067 ceph-mon[30939]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T23:26:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:25:59 smithi092 ceph-mon[34223]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T23:26:00.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:25:59 smithi145 ceph-mon[34397]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T23:26:00.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 -- 172.21.15.67:0/2758960192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10a800 msgr2=0x7fb78c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 --2- 172.21.15.67:0/2758960192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10a800 0x7fb78c10ac40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb778009900 tx=0x7fb77802a410 comp rx=0 tx=0).stop 2024-09-10T23:26:00.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 -- 172.21.15.67:0/2758960192 shutdown_connections 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 --2- 172.21.15.67:0/2758960192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10b180 0x7fb78c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 --2- 172.21.15.67:0/2758960192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10a800 0x7fb78c10ac40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 --2- 172.21.15.67:0/2758960192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 -- 172.21.15.67:0/2758960192 >> 172.21.15.67:0/2758960192 conn(0x7fb78c100840 msgr2=0x7fb78c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 -- 172.21.15.67:0/2758960192 shutdown_connections 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.414+0000 7fb791200640 1 -- 172.21.15.67:0/2758960192 wait complete. 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.415+0000 7fb791200640 1 Processor -- start 2024-09-10T23:26:00.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.415+0000 7fb791200640 1 -- start start 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 0x7fb78c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c114cd0 con 0x7fb78c10b180 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c114b50 con 0x7fb78c10a800 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb791200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c114e50 con 0x7fb78c104df0 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41150/0 (socket says 172.21.15.67:41150) 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb78be00640 1 -- 172.21.15.67:0/2670132880 learned_addr learned my addr 172.21.15.67:0/2670132880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb78b400640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.416+0000 7fb78aa00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 0x7fb78c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 -- 172.21.15.67:0/2670132880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 msgr2=0x7fb78c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 -- 172.21.15.67:0/2670132880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 msgr2=0x7fb78c19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 0x7fb78c19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 -- 172.21.15.67:0/2670132880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c1a1170 con 0x7fb78c10b180 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78b400640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78aa00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 0x7fb78c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb78be00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb78400eb10 tx=0x7fb78400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb783e00640 1 -- 172.21.15.67:0/2670132880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784004b20 con 0x7fb78c10b180 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb783e00640 1 -- 172.21.15.67:0/2670132880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb784004cc0 con 0x7fb78c10b180 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb783e00640 1 -- 172.21.15.67:0/2670132880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784010690 con 0x7fb78c10b180 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c1a1460 con 0x7fb78c10b180 2024-09-10T23:26:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.417+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c1a7b20 con 0x7fb78c10b180 2024-09-10T23:26:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.421+0000 7fb783e00640 1 -- 172.21.15.67:0/2670132880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb784005020 con 0x7fb78c10b180 2024-09-10T23:26:00.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.422+0000 7fb783e00640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 0x7fb7700b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.422+0000 7fb783e00640 1 -- 172.21.15.67:0/2670132880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9595+0+0 (secure 0 0 0) 0x7fb7840e6a60 con 0x7fb78c10b180 2024-09-10T23:26:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.422+0000 7fb78b400640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 0x7fb7700b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.423+0000 7fb78b400640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 0x7fb7700b7a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00bfd0 tx=0x7fb77c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 Processor -- start 2024-09-10T23:26:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 -- start start 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72114af0 0x562c72120540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562c7208b290 con 0x562c72120a80 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562c7207a6c0 con 0x562c721141d0 2024-09-10T23:26:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.436+0000 7fb7951bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562c7207e4d0 con 0x562c72114af0 2024-09-10T23:26:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:43912/0 (socket says 172.21.15.67:43912) 2024-09-10T23:26:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41152/0 (socket says 172.21.15.67:41152) 2024-09-10T23:26:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 -- 172.21.15.67:0/2949030474 learned_addr learned my addr 172.21.15.67:0/2949030474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78aa00640 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72114af0 0x562c72120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 -- 172.21.15.67:0/2949030474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72114af0 msgr2=0x562c72120540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72114af0 0x562c72120540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 -- 172.21.15.67:0/2949030474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 msgr2=0x562c72122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78b400640 1 -- 172.21.15.67:0/2949030474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c72007460 con 0x562c721141d0 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.437+0000 7fb78be00640 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.438+0000 7fb78b400640 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011800 tx=0x7fb77c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8c2fbbdaf56f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.438+0000 7fb780c00640 1 -- 172.21.15.67:0/2949030474 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c01cac0 con 0x562c721141d0 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.438+0000 7fb780c00640 1 -- 172.21.15.67:0/2949030474 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb77c011e40 con 0x562c721141d0 2024-09-10T23:26:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.438+0000 7fb780c00640 1 -- 172.21.15.67:0/2949030474 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c01d3b0 con 0x562c721141d0 2024-09-10T23:26:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 -- 172.21.15.67:0/2949030474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 msgr2=0x562c721145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011800 tx=0x7fb77c013000 comp rx=0 tx=0).stop 2024-09-10T23:26:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 -- 172.21.15.67:0/2949030474 shutdown_connections 2024-09-10T23:26:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c72122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72114af0 0x562c72120540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2949030474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c721145b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 -- 172.21.15.67:0/2949030474 >> 172.21.15.67:0/2949030474 conn(0x562c72112000 msgr2=0x562c721131a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 -- 172.21.15.67:0/2949030474 shutdown_connections 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.439+0000 7fb7951bf740 1 -- 172.21.15.67:0/2949030474 wait complete. 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 -- start start 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 Processor -- start 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 0x562c7211da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562c71edbbb0 con 0x562c72120a80 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562c720156c0 con 0x562c72114af0 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562c72123730 con 0x562c721141d0 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb7951bf740 10 client.0 _ll_register_callbacks cb 0x562c72030490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:00.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.440+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41168/0 (socket says 172.21.15.67:41168) 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 -- 172.21.15.67:0/387630049 learned_addr learned my addr 172.21.15.67:0/387630049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78b400640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 0x562c7211da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78aa00640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 -- 172.21.15.67:0/387630049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 msgr2=0x562c7211da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 0x562c7211da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 -- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 msgr2=0x562c7211df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 -- 172.21.15.67:0/387630049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c71dd8b00 con 0x562c72120a80 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78aa00640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:26:00.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78b400640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 0x562c7211da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb78be00640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ac9e0 tx=0x7fb7840ac8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20128b87c2c34fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.441+0000 7fb775600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840b5d50 con 0x562c72120a80 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9595+0+0 (secure 0 0 0) 0x7fb7840abcb0 con 0x562c72120a80 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c7208d1e0 con 0x562c72120a80 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb7840c7050 con 0x562c72120a80 2024-09-10T23:26:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb7951bf740 10 client.16548 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562c71e4f960 con 0x562c72120a80 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562c7208d1e0 con 0x562c72120a80 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.442+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb7840ab720 con 0x562c72120a80 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 10 client.16548 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 10 client.16548 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 20 client.16548 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c72130610 0x562c72132a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 4 client.16548 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c72088c90 con 0x562c72130610 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.443+0000 7fb78b400640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c72130610 0x562c72132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.444+0000 7fb78b400640 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c72130610 0x562c72132a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85bb8fc7a301ce8 server_cookie=efbca140d30ea7d6 in_seq=0 out_seq=0 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.444+0000 7fb775600640 10 client.16548 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb775600640 1 -- 172.21.15.67:0/387630049 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c72088c90 con 0x562c72130610 2024-09-10T23:26:00.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb775600640 10 client.16548 handle_command_reply: tid=0 2024-09-10T23:26:00.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 client.16548 shutdown 2024-09-10T23:26:00.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 msgr2=0x562c7211e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ac9e0 tx=0x7fb7840ac8e0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 shutdown_connections 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c72130610 0x562c72132a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72120a80 0x562c7211e4a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb775600640 0 client.16548 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 --2- 172.21.15.67:0/387630049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c721141d0 0x562c7211da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 >> 172.21.15.67:0/387630049 conn(0x562c72112000 msgr2=0x562c72117b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.445+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 shutdown_connections 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.446+0000 7fb7951bf740 1 -- 172.21.15.67:0/387630049 wait complete. 2024-09-10T23:26:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.446+0000 7fb7951bf740 20 client.16548 trim_cache size 0 max 16384 2024-09-10T23:26:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 Processor -- start 2024-09-10T23:26:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 -- start start 2024-09-10T23:26:00.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c7211b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1b4800 con 0x562c721141d0 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1b2070 con 0x562c72114af0 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.464+0000 7fb7951bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1afee0 con 0x562c72120a80 2024-09-10T23:26:00.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41184/0 (socket says 172.21.15.67:41184) 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 -- 172.21.15.67:0/1097510808 learned_addr learned my addr 172.21.15.67:0/1097510808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78aa00640 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78be00640 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c7211b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 -- 172.21.15.67:0/1097510808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 msgr2=0x562c7211b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c7211b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 -- 172.21.15.67:0/1097510808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 msgr2=0x562c7211ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.465+0000 7fb78b400640 1 -- 172.21.15.67:0/1097510808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c72111e30 con 0x562c721141d0 2024-09-10T23:26:00.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.466+0000 7fb78b400640 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011800 tx=0x7fb77c011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29c76b8f1fe6500 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.466+0000 7fb777e00640 1 -- 172.21.15.67:0/1097510808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c01ea70 con 0x562c721141d0 2024-09-10T23:26:00.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.466+0000 7fb777e00640 1 -- 172.21.15.67:0/1097510808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb77c005040 con 0x562c721141d0 2024-09-10T23:26:00.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.466+0000 7fb777e00640 1 -- 172.21.15.67:0/1097510808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c01caa0 con 0x562c721141d0 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 -- 172.21.15.67:0/1097510808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 msgr2=0x562c7211e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011800 tx=0x7fb77c011560 comp rx=0 tx=0).stop 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 -- 172.21.15.67:0/1097510808 shutdown_connections 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c7211b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c72114af0 0x562c7211ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 --2- 172.21.15.67:0/1097510808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c721141d0 0x562c7211e100 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 -- 172.21.15.67:0/1097510808 >> 172.21.15.67:0/1097510808 conn(0x562c72112000 msgr2=0x562c7211d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 -- 172.21.15.67:0/1097510808 shutdown_connections 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.467+0000 7fb7951bf740 1 -- 172.21.15.67:0/1097510808 wait complete. 2024-09-10T23:26:00.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.468+0000 7fb7951bf740 1 -- start start 2024-09-10T23:26:00.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.468+0000 7fb7951bf740 1 Processor -- start 2024-09-10T23:26:00.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c72129050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 0x562c72123980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c72123ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1b0530 con 0x562c72114af0 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1b19a0 con 0x562c721141d0 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1b0c00 con 0x562c72120a80 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb7951bf740 10 client.0 _ll_register_callbacks cb 0x7fb784005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb78b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c72129050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb78b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c72129050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:43936/0 (socket says 172.21.15.67:43936) 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb78b400640 1 -- 172.21.15.67:0/2009542043 learned_addr learned my addr 172.21.15.67:0/2009542043 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:00.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.469+0000 7fb78be00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c72123ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.470+0000 7fb78be00640 1 -- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 msgr2=0x562c72129050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.470+0000 7fb78be00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c72129050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.472+0000 7fb78aa00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 0x562c72123980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.470+0000 7fb78be00640 1 -- 172.21.15.67:0/2009542043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 msgr2=0x562c72123980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.472+0000 7fb78be00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 0x562c72123980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.472+0000 7fb78be00640 1 -- 172.21.15.67:0/2009542043 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c721245d0 con 0x562c72120a80 2024-09-10T23:26:00.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.472+0000 7fb78aa00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 0x562c72123980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:26:00.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.474+0000 7fb78be00640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c72123ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7840a9cc0 tx=0x7fb7840b5d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538805d740505313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:00.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.474+0000 7fb775600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840cb020 con 0x562c72120a80 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9595+0+0 (secure 0 0 0) 0x7fb7840a83a0 con 0x562c72120a80 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c005040 con 0x562c72120a80 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb7840a88f0 con 0x562c72120a80 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb7951bf740 10 client.26248 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:00.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562c7219e470 con 0x562c72120a80 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb77c005040 con 0x562c72120a80 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb7840a85a0 con 0x562c72120a80 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb7951bf740 10 client.26248 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.475+0000 7fb7951bf740 10 client.26248 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.476+0000 7fb7951bf740 20 client.26248 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.476+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c7219eb60 0x562c721a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:00.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.476+0000 7fb7951bf740 4 client.26248 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7fa737a0", "release": true, "await": true}] 2024-09-10T23:26:00.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.476+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7fa737a0", "release": true, "await": true}) -- 0x562c720860c0 con 0x562c7219eb60 2024-09-10T23:26:00.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.476+0000 7fb78b400640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c7219eb60 0x562c721a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:00.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.477+0000 7fb78b400640 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c7219eb60 0x562c721a0f80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525a7aa79aec85fd server_cookie=21cb0f222d716bca in_seq=0 out_seq=0 2024-09-10T23:26:00.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.477+0000 7fb775600640 10 client.26248 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:00.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.506+0000 7fb775600640 1 -- 172.21.15.67:0/2009542043 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562c720860c0 con 0x562c7219eb60 2024-09-10T23:26:00.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.506+0000 7fb775600640 10 client.26248 handle_command_reply: tid=0 2024-09-10T23:26:00.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.506+0000 7fb7951bf740 1 client.26248 shutdown 2024-09-10T23:26:00.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 msgr2=0x562c72123ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c72123ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7840a9cc0 tx=0x7fb7840b5d80 comp rx=0 tx=0).stop 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 shutdown_connections 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x562c7219eb60 0x562c721a0f80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c72120a80 0x562c72123ec0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c72114af0 0x562c72123980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb775600640 0 client.26248 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 --2- 172.21.15.67:0/2009542043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c721141d0 0x562c72129050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 >> 172.21.15.67:0/2009542043 conn(0x562c72112000 msgr2=0x562c721110c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 shutdown_connections 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 1 -- 172.21.15.67:0/2009542043 wait complete. 2024-09-10T23:26:00.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.507+0000 7fb7951bf740 20 client.26248 trim_cache size 0 max 16384 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 41, 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 80, 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "7fa737a0": { 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 80, 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:00.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:00.511 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.508+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 msgr2=0x7fb7700b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.508+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 0x7fb7700b7a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00bfd0 tx=0x7fb77c00b390 comp rx=0 tx=0).stop 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.508+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 msgr2=0x7fb78c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.508+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb78400eb10 tx=0x7fb78400c2d0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 shutdown_connections 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb7700b5580 0x7fb7700b7a40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb78c10b180 0x7fb78c1a09a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb78c10a800 0x7fb78c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 --2- 172.21.15.67:0/2670132880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb78c104df0 0x7fb78c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:00.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 >> 172.21.15.67:0/2670132880 conn(0x7fb78c100840 msgr2=0x7fb78c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:00.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 shutdown_connections 2024-09-10T23:26:00.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:00.509+0000 7fb791200640 1 -- 172.21.15.67:0/2670132880 wait complete. 2024-09-10T23:26:01.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:01 smithi067 ceph-mon[30939]: pgmap v1240: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.9 MiB/s wr, 1.24k op/s 2024-09-10T23:26:01.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:01 smithi145 ceph-mon[34397]: pgmap v1240: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.9 MiB/s wr, 1.24k op/s 2024-09-10T23:26:01.467 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7fa737a0', total seconds elapsed: 44.5 2024-09-10T23:26:01.467 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T23:26:01.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:26:01.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:01 smithi092 ceph-mon[34223]: pgmap v1240: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.9 MiB/s wr, 1.24k op/s 2024-09-10T23:26:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:26:04.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:03 smithi092 ceph-mon[34223]: pgmap v1241: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 723 op/s 2024-09-10T23:26:04.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:03 smithi067 ceph-mon[30939]: pgmap v1241: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 723 op/s 2024-09-10T23:26:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:03 smithi145 ceph-mon[34397]: pgmap v1241: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 723 op/s 2024-09-10T23:26:04.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:04 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:26:04.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:04 smithi067 ceph-mon[30939]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T23:26:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:04 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:26:05.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:04 smithi092 ceph-mon[34223]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T23:26:05.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:04 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:26:05.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:04 smithi145 ceph-mon[34397]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T23:26:06.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: pgmap v1243: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-10T23:26:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T23:26:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:05 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: pgmap v1243: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-10T23:26:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T23:26:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:05 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: pgmap v1243: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-10T23:26:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T23:26:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:05 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:06.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 -- 172.21.15.67:0/1988595913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 msgr2=0x7f52f80a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:06.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 --2- 172.21.15.67:0/1988595913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f80a23e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009930 tx=0x7f52f402a420 comp rx=0 tx=0).stop 2024-09-10T23:26:06.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 -- 172.21.15.67:0/1988595913 shutdown_connections 2024-09-10T23:26:06.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 --2- 172.21.15.67:0/1988595913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f80a23e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:06.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 --2- 172.21.15.67:0/1988595913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 0x7f52f809b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 --2- 172.21.15.67:0/1988595913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 0x7f52f80052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 -- 172.21.15.67:0/1988595913 >> 172.21.15.67:0/1988595913 conn(0x7f52f800a160 msgr2=0x7f52f800c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.602+0000 7f5307000640 1 -- 172.21.15.67:0/1988595913 shutdown_connections 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.603+0000 7f5307000640 1 -- 172.21.15.67:0/1988595913 wait complete. 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.604+0000 7f5307000640 1 Processor -- start 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.604+0000 7f5307000640 1 -- start start 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.605+0000 7f5307000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 0x7f52f813ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.605+0000 7f5307000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 0x7f52f813d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.605+0000 7f5307000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:06.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5307000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80a57d0 con 0x7f52f809b260 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5307000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80a5650 con 0x7f52f809bbc0 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5307000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80a5950 con 0x7f52f8004ee0 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5306600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5306600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:43956/0 (socket says 172.21.15.67:43956) 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.606+0000 7f5306600640 1 -- 172.21.15.67:0/2772992589 learned_addr learned my addr 172.21.15.67:0/2772992589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.607+0000 7f5305c00640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 0x7f52f813ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.607+0000 7f5305200640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 0x7f52f813d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.608+0000 7f5306600640 1 -- 172.21.15.67:0/2772992589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 msgr2=0x7f52f813ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.608+0000 7f5306600640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 0x7f52f813ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.608+0000 7f5306600640 1 -- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 msgr2=0x7f52f813d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.608+0000 7f5306600640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 0x7f52f813d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:06.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.608+0000 7f5306600640 1 -- 172.21.15.67:0/2772992589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f8142f60 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.609+0000 7f5306600640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52f402aa10 tx=0x7f52f4003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.613+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f403f070 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.613+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52f403bd00 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.613+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4035030 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.614+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f81431f0 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.614+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f80948b0 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.617+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f8096cd0 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.621+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f52f4031400 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.625+0000 7f52fe000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 0x7f52e80b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.626+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9607+0+0 (secure 0 0 0) 0x7f52f4107300 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.626+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52f4049030 con 0x7f52f809bbc0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.627+0000 7f5305c00640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 0x7f52e80b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:06.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:06.628+0000 7f5305c00640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 0x7f52e80b7ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f53000669a0 tx=0x7f5300068a30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:07.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:06 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:26:07.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:06 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:26:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:06 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:26:07.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:06 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:26:07.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.102+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52f80052c0 con 0x7f52f809bbc0 2024-09-10T23:26:07.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.104+0000 7f52fe000640 1 -- 172.21.15.67:0/2772992589 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11905 (secure 0 0 0) 0x7f52f40cda60 con 0x7f52f809bbc0 2024-09-10T23:26:07.111 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:26:07.111 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"btime":"2024-09-10T23:25:31:756758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:25:31.060860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:26:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-09-10T23:26:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.110+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 msgr2=0x7f52e80b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.110+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 0x7f52e80b7ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f53000669a0 tx=0x7f5300068a30 comp rx=0 tx=0).stop 2024-09-10T23:26:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.110+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 msgr2=0x7f52f8142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:07.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.110+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52f402aa10 tx=0x7f52f4003cb0 comp rx=0 tx=0).stop 2024-09-10T23:26:07.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 shutdown_connections 2024-09-10T23:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f52e80b5620 0x7f52e80b7ae0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f809bbc0 0x7f52f8142790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52f809b260 0x7f52f813d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 --2- 172.21.15.67:0/2772992589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52f8004ee0 0x7f52f813ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.111+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 >> 172.21.15.67:0/2772992589 conn(0x7f52f800a160 msgr2=0x7f52f8099310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.113+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 shutdown_connections 2024-09-10T23:26:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:07.113+0000 7f5307000640 1 -- 172.21.15.67:0/2772992589 wait complete. 2024-09-10T23:26:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:06 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi092 to 176.4M 2024-09-10T23:26:07.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:06 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi092 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-09-10T23:26:08.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:07 smithi092 ceph-mon[34223]: pgmap v1245: 129 pgs: 10 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 91 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:08.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:07 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2772992589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:07 smithi067 ceph-mon[30939]: pgmap v1245: 129 pgs: 10 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 91 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:07 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2772992589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:07 smithi145 ceph-mon[34397]: pgmap v1245: 129 pgs: 10 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 91 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:07 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2772992589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:08.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:08.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:08.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:08.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:08 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:08 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:08 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:10.005 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:09 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:26:10.005 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:09 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:26:10.005 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:09 smithi145 ceph-mon[34397]: pgmap v1246: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:10.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:09 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:26:10.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:09 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:26:10.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:09 smithi067 ceph-mon[30939]: pgmap v1246: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:09 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi067 to 176.4M 2024-09-10T23:26:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:09 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi067 to 185044787: error parsing value: Value '185044787' is below minimum 939524096 2024-09-10T23:26:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:09 smithi092 ceph-mon[34223]: pgmap v1246: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 1.08k op/s 2024-09-10T23:26:12.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:11 smithi092 ceph-mon[34223]: pgmap v1247: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 1.70k op/s 2024-09-10T23:26:12.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:11 smithi067 ceph-mon[30939]: pgmap v1247: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 1.70k op/s 2024-09-10T23:26:12.536 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:11 smithi145 ceph-mon[34397]: pgmap v1247: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.7 MiB/s wr, 1.70k op/s 2024-09-10T23:26:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:13 smithi145 ceph-mon[34397]: pgmap v1248: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-09-10T23:26:14.273 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:13 smithi092 ceph-mon[34223]: pgmap v1248: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-09-10T23:26:14.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:13 smithi067 ceph-mon[30939]: pgmap v1248: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-09-10T23:26:14.634 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T23:26:14.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T23:26:15.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:14 smithi067 ceph-mon[30939]: pgmap v1249: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 2.11k op/s 2024-09-10T23:26:15.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:14 smithi145 ceph-mon[34397]: pgmap v1249: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 2.11k op/s 2024-09-10T23:26:15.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:14 smithi092 ceph-mon[34223]: pgmap v1249: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 2.11k op/s 2024-09-10T23:26:17.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:26:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:17 smithi092 ceph-mon[34223]: pgmap v1250: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-09-10T23:26:18.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:17 smithi067 ceph-mon[30939]: pgmap v1250: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-09-10T23:26:18.096 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:17 smithi145 ceph-mon[34397]: pgmap v1250: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: pgmap v1251: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 2.11k op/s 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:19.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:26:19.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:26:19.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:19 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: pgmap v1251: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 2.11k op/s 2024-09-10T23:26:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:26:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:19 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: pgmap v1251: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 2.11k op/s 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:26:20.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:26:20.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:26:20.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:19 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:26:20.758 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:20 smithi067 ceph-mon[30939]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:26:20.758 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:20 smithi067 ceph-mon[30939]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:26:20.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.862+0000 7fb28f200640 1 -- 172.21.15.67:0/3634367064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290102220 msgr2=0x7fb2901066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.862+0000 7fb28f200640 1 --2- 172.21.15.67:0/3634367064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290102220 0x7fb2901066f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009860 tx=0x7fb27c02a360 comp rx=0 tx=0).stop 2024-09-10T23:26:20.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.868+0000 7fb28f200640 1 -- 172.21.15.67:0/3634367064 shutdown_connections 2024-09-10T23:26:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.868+0000 7fb28f200640 1 --2- 172.21.15.67:0/3634367064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290106d40 0x7fb290107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.868+0000 7fb28f200640 1 --2- 172.21.15.67:0/3634367064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290102220 0x7fb2901066f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.868+0000 7fb28f200640 1 --2- 172.21.15.67:0/3634367064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290101900 0x7fb290101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.868+0000 7fb28f200640 1 -- 172.21.15.67:0/3634367064 >> 172.21.15.67:0/3634367064 conn(0x7fb2900fd5b0 msgr2=0x7fb2900ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.869+0000 7fb28f200640 1 -- 172.21.15.67:0/3634367064 shutdown_connections 2024-09-10T23:26:20.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.869+0000 7fb28f200640 1 -- 172.21.15.67:0/3634367064 wait complete. 2024-09-10T23:26:20.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.873+0000 7fb28f200640 1 Processor -- start 2024-09-10T23:26:20.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.873+0000 7fb28f200640 1 -- start start 2024-09-10T23:26:20.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.876+0000 7fb28f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28f200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290102220 0x7fb29019fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290106d40 0x7fb2901a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb290119060 con 0x7fb290106d40 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28f200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb290118ee0 con 0x7fb290102220 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901191e0 con 0x7fb290101900 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43484/0 (socket says 172.21.15.67:43484) 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.877+0000 7fb28de00640 1 -- 172.21.15.67:0/170724285 learned_addr learned my addr 172.21.15.67:0/170724285 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28d400640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290102220 0x7fb29019fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28e800640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290106d40 0x7fb2901a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28de00640 1 -- 172.21.15.67:0/170724285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290102220 msgr2=0x7fb29019fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28de00640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290102220 0x7fb29019fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28de00640 1 -- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290106d40 msgr2=0x7fb2901a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28de00640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290106d40 0x7fb2901a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.878+0000 7fb28de00640 1 -- 172.21.15.67:0/170724285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2901a5600 con 0x7fb290101900 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.879+0000 7fb28de00640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e9c0 tx=0x7fb28000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.880+0000 7fb286a00640 1 -- 172.21.15.67:0/170724285 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000ccc0 con 0x7fb290101900 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.880+0000 7fb286a00640 1 -- 172.21.15.67:0/170724285 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800054b0 con 0x7fb290101900 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.880+0000 7fb286a00640 1 -- 172.21.15.67:0/170724285 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280010430 con 0x7fb290101900 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.880+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2901a58f0 con 0x7fb290101900 2024-09-10T23:26:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.880+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2901abfb0 con 0x7fb290101900 2024-09-10T23:26:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.887+0000 7fb286a00640 1 -- 172.21.15.67:0/170724285 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb280004e70 con 0x7fb290101900 2024-09-10T23:26:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.890+0000 7fb286a00640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 0x7fb2740b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.891+0000 7fb28d400640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 0x7fb2740b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.891+0000 7fb286a00640 1 -- 172.21.15.67:0/170724285 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9607+0+0 (secure 0 0 0) 0x7fb2800e67b0 con 0x7fb290101900 2024-09-10T23:26:20.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.892+0000 7fb28d400640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 0x7fb2740b7b10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009460 tx=0x7fb27c009390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.902+0000 7fb297c1d740 1 Processor -- start 2024-09-10T23:26:20.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.904+0000 7fb297c1d740 1 -- start start 2024-09-10T23:26:20.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.904+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564541571c10 0x56454158e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56454158ebb0 0x564541590fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb297c1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645414e6ca0 con 0x56454158ebb0 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb297c1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645414d60d0 con 0x564541571c10 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb297c1d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5645414d9ee0 con 0x5645415712f0 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.905+0000 7fb28d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564541571c10 0x56454158e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.906+0000 7fb28e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56454158ebb0 0x564541590fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.906+0000 7fb28de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.906+0000 7fb28de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43498/0 (socket says 172.21.15.67:43498) 2024-09-10T23:26:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.906+0000 7fb28de00640 1 -- 172.21.15.67:0/789472253 learned_addr learned my addr 172.21.15.67:0/789472253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 -- 172.21.15.67:0/789472253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564541571c10 msgr2=0x56454158e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564541571c10 0x56454158e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 -- 172.21.15.67:0/789472253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56454158ebb0 msgr2=0x564541590fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56454158ebb0 0x564541590fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 -- 172.21.15.67:0/789472253 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564541488400 con 0x5645415712f0 2024-09-10T23:26:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.907+0000 7fb28de00640 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb2800ac5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8927beb47c15d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.908+0000 7fb27b400640 1 -- 172.21.15.67:0/789472253 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800b10d0 con 0x5645415712f0 2024-09-10T23:26:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.908+0000 7fb27b400640 1 -- 172.21.15.67:0/789472253 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800046c0 con 0x5645415712f0 2024-09-10T23:26:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.908+0000 7fb27b400640 1 -- 172.21.15.67:0/789472253 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800b12d0 con 0x5645415712f0 2024-09-10T23:26:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 -- 172.21.15.67:0/789472253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 msgr2=0x5645415716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb2800ac5c0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 -- 172.21.15.67:0/789472253 shutdown_connections 2024-09-10T23:26:20.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56454158ebb0 0x564541590fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564541571c10 0x56454158e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 --2- 172.21.15.67:0/789472253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x5645415716d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.909+0000 7fb297c1d740 1 -- 172.21.15.67:0/789472253 >> 172.21.15.67:0/789472253 conn(0x56454156f120 msgr2=0x5645415702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.910+0000 7fb297c1d740 1 -- 172.21.15.67:0/789472253 shutdown_connections 2024-09-10T23:26:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.910+0000 7fb297c1d740 1 -- 172.21.15.67:0/789472253 wait complete. 2024-09-10T23:26:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.911+0000 7fb297c1d740 1 -- start start 2024-09-10T23:26:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.911+0000 7fb297c1d740 1 Processor -- start 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645415712f0 0x56454158e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x56454158beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5645413377a0 con 0x5645415712f0 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56454147ba10 con 0x56454158ebb0 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564541591860 con 0x564541571c10 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.912+0000 7fb297c1d740 10 client.0 _ll_register_callbacks cb 0x56454148bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.913+0000 7fb28d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.913+0000 7fb28d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43508/0 (socket says 172.21.15.67:43508) 2024-09-10T23:26:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.913+0000 7fb28d400640 1 -- 172.21.15.67:0/1675993412 learned_addr learned my addr 172.21.15.67:0/1675993412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.913+0000 7fb28de00640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645415712f0 0x56454158e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 -- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 msgr2=0x56454158beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x56454158beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 -- 172.21.15.67:0/1675993412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645415712f0 msgr2=0x56454158e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645415712f0 0x56454158e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 -- 172.21.15.67:0/1675993412 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564541243b90 con 0x564541571c10 2024-09-10T23:26:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb28d400640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009860 tx=0x7fb27c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2901a7f1b36bcdab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.914+0000 7fb26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:26:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.915+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c04e070 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.915+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9607+0+0 (secure 0 0 0) 0x7fb27c005220 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.915+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645414e8bf0 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.915+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb27c037040 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.916+0000 7fb297c1d740 10 client.26263 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.916+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56454159d170 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.916+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5645414e8bf0 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.916+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb27c049070 con 0x564541571c10 2024-09-10T23:26:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 10 client.26263 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:20.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 10 client.26263 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:20.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 20 client.26263 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:20.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454159e580 0x5645415a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 4 client.26263 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:26:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.917+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645414e46a0 con 0x56454159e580 2024-09-10T23:26:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.921+0000 7fb28de00640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454159e580 0x5645415a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.922+0000 7fb26fe00640 10 client.26263 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.923+0000 7fb28de00640 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454159e580 0x5645415a0a10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fee905f8f0338e server_cookie=ded921f007255619 in_seq=0 out_seq=0 2024-09-10T23:26:20.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.927+0000 7fb26fe00640 1 -- 172.21.15.67:0/1675993412 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645414e46a0 con 0x56454159e580 2024-09-10T23:26:20.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.927+0000 7fb26fe00640 10 client.26263 handle_command_reply: tid=0 2024-09-10T23:26:20.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.927+0000 7fb297c1d740 1 client.26263 shutdown 2024-09-10T23:26:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.927+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 msgr2=0x56454158b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.927+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009860 tx=0x7fb27c005f90 comp rx=0 tx=0).stop 2024-09-10T23:26:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 shutdown_connections 2024-09-10T23:26:20.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454159e580 0x5645415a0a10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x56454158beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564541571c10 0x56454158b8d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 --2- 172.21.15.67:0/1675993412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5645415712f0 0x56454158e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 >> 172.21.15.67:0/1675993412 conn(0x56454156f120 msgr2=0x56454156e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb26fe00640 0 client.26263 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 shutdown_connections 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 1 -- 172.21.15.67:0/1675993412 wait complete. 2024-09-10T23:26:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.928+0000 7fb297c1d740 20 client.26263 trim_cache size 0 max 16384 2024-09-10T23:26:20.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.969+0000 7fb297c1d740 1 Processor -- start 2024-09-10T23:26:20.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.969+0000 7fb297c1d740 1 -- start start 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 0x56454159aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x56454158fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x56454158b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901b04f0 con 0x564541571c10 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901add60 con 0x56454156f120 2024-09-10T23:26:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.970+0000 7fb297c1d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901066f0 con 0x5645415712f0 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.971+0000 7fb28de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 0x56454159aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.971+0000 7fb28d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x56454158fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.971+0000 7fb28d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x56454158fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:43522/0 (socket says 172.21.15.67:43522) 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.971+0000 7fb28d400640 1 -- 172.21.15.67:0/3730035724 learned_addr learned my addr 172.21.15.67:0/3730035724 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.972+0000 7fb28e800640 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x56454158b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.972+0000 7fb28de00640 1 -- 172.21.15.67:0/3730035724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 msgr2=0x56454158fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.973+0000 7fb28de00640 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x56454158fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.973+0000 7fb28de00640 1 -- 172.21.15.67:0/3730035724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 msgr2=0x56454158b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.973+0000 7fb28de00640 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x56454158b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.973+0000 7fb28de00640 1 -- 172.21.15.67:0/3730035724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564541570930 con 0x56454156f120 2024-09-10T23:26:20.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.976+0000 7fb28de00640 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 0x56454159aa90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564541591530 tx=0x7fb280009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593fea97018676cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.976+0000 7fb27aa00640 1 -- 172.21.15.67:0/3730035724 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800acef0 con 0x56454156f120 2024-09-10T23:26:20.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.977+0000 7fb27aa00640 1 -- 172.21.15.67:0/3730035724 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800b5d80 con 0x56454156f120 2024-09-10T23:26:20.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.977+0000 7fb27aa00640 1 -- 172.21.15.67:0/3730035724 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800cfb40 con 0x56454156f120 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 -- 172.21.15.67:0/3730035724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 msgr2=0x56454159aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 0x56454159aa90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564541591530 tx=0x7fb280009740 comp rx=0 tx=0).stop 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 -- 172.21.15.67:0/3730035724 shutdown_connections 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x56454158b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x56454158fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3730035724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454156f120 0x56454159aa90 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564541591530 tx=0x7fb280009740 comp rx=0 tx=0).stop 2024-09-10T23:26:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 -- 172.21.15.67:0/3730035724 >> 172.21.15.67:0/3730035724 conn(0x564541590120 msgr2=0x56454158cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:20.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 -- 172.21.15.67:0/3730035724 shutdown_connections 2024-09-10T23:26:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.978+0000 7fb297c1d740 1 -- 172.21.15.67:0/3730035724 wait complete. 2024-09-10T23:26:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.979+0000 7fb297c1d740 1 -- start start 2024-09-10T23:26:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.979+0000 7fb297c1d740 1 Processor -- start 2024-09-10T23:26:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.982+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x564541596cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.982+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x564541597210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.982+0000 7fb297c1d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x564541593ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb28e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x564541593ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb28e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x564541593ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:50480/0 (socket says 172.21.15.67:50480) 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb28e800640 1 -- 172.21.15.67:0/3217729593 learned_addr learned my addr 172.21.15.67:0/3217729593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb297c1d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ac290 con 0x564541571c10 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ad690 con 0x56454158ebb0 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901ac8f0 con 0x5645415712f0 2024-09-10T23:26:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.983+0000 7fb297c1d740 10 client.0 _ll_register_callbacks cb 0x7fb27c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.984+0000 7fb28de00640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x564541596cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.984+0000 7fb28d400640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x564541597210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.985+0000 7fb28de00640 1 -- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 msgr2=0x564541593ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.985+0000 7fb28de00640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x564541593ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.985+0000 7fb28de00640 1 -- 172.21.15.67:0/3217729593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 msgr2=0x564541597210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.985+0000 7fb28de00640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x564541597210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.985+0000 7fb28de00640 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564541594160 con 0x5645415712f0 2024-09-10T23:26:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.986+0000 7fb28de00640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x564541596cd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56454156f910 tx=0x7fb2800094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1dc9b82ad8d6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.986+0000 7fb26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:26:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.986+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800b12d0 con 0x5645415712f0 2024-09-10T23:26:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.986+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9607+0+0 (secure 0 0 0) 0x7fb2800a8a60 con 0x5645415712f0 2024-09-10T23:26:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.986+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2800b5d80 con 0x5645415712f0 2024-09-10T23:26:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.991+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb2800acef0 con 0x5645415712f0 2024-09-10T23:26:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 10 client.26269 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56454160be70 con 0x5645415712f0 2024-09-10T23:26:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2800b5d80 con 0x5645415712f0 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb2800aba50 con 0x5645415712f0 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 10 client.26269 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 10 client.26269 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 20 client.26269 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.992+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454160c0a0 0x56454160e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.993+0000 7fb297c1d740 4 client.26269 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T23:26:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.993+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5645414e1ad0 con 0x56454160c0a0 2024-09-10T23:26:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.995+0000 7fb28d400640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454160c0a0 0x56454160e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.996+0000 7fb28d400640 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454160c0a0 0x56454160e490 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474eff15095ecad0 server_cookie=47bfdbec6b00e80f in_seq=0 out_seq=0 2024-09-10T23:26:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:20.996+0000 7fb26fe00640 10 client.26269 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:21.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:20 smithi092 ceph-mon[34223]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:26:21.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:20 smithi092 ceph-mon[34223]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:26:21.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:20 smithi145 ceph-mon[34397]: Adjusting osd_memory_target on smithi145 to 176.4M 2024-09-10T23:26:21.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:20 smithi145 ceph-mon[34397]: Unable to set osd_memory_target on smithi145 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-10T23:26:21.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:21 smithi067 ceph-mon[30939]: pgmap v1252: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:26:22.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:21 smithi092 ceph-mon[34223]: pgmap v1252: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:26:22.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:21 smithi145 ceph-mon[34397]: pgmap v1252: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-09-10T23:26:24.029 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:23 smithi092 ceph-mon[34223]: pgmap v1253: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T23:26:24.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:23 smithi067 ceph-mon[30939]: pgmap v1253: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T23:26:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:23 smithi145 ceph-mon[34397]: pgmap v1253: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T23:26:25.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:25 smithi092 ceph-mon[34223]: pgmap v1254: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.1 MiB/s wr, 2.30k op/s 2024-09-10T23:26:25.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:25 smithi067 ceph-mon[30939]: pgmap v1254: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.1 MiB/s wr, 2.30k op/s 2024-09-10T23:26:25.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:25 smithi145 ceph-mon[34397]: pgmap v1254: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.1 MiB/s wr, 2.30k op/s 2024-09-10T23:26:27.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.566+0000 7fb26fe00640 1 -- 172.21.15.67:0/3217729593 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5645414e1ad0 con 0x56454160c0a0 2024-09-10T23:26:27.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.566+0000 7fb26fe00640 10 client.26269 handle_command_reply: tid=0 2024-09-10T23:26:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.567+0000 7fb297c1d740 1 client.26269 shutdown 2024-09-10T23:26:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 msgr2=0x564541596cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x564541596cd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56454156f910 tx=0x7fb2800094c0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 shutdown_connections 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56454160c0a0 0x56454160e490 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb26fe00640 0 client.26269 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56454158ebb0 0x564541593ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564541571c10 0x564541597210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 --2- 172.21.15.67:0/3217729593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645415712f0 0x564541596cd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.568+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 >> 172.21.15.67:0/3217729593 conn(0x564541590120 msgr2=0x56454158d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.569+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 shutdown_connections 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.569+0000 7fb297c1d740 1 -- 172.21.15.67:0/3217729593 wait complete. 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.569+0000 7fb297c1d740 20 client.26269 trim_cache size 0 max 16384 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:26:27.570 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 41, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 82, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "c91ff02b": { 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 82, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:26:27.571 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:27.572 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.570+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 msgr2=0x7fb2740b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.570+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 0x7fb2740b7b10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009460 tx=0x7fb27c009390 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.570+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 msgr2=0x7fb29019f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.570+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e9c0 tx=0x7fb28000edc0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 shutdown_connections 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fb2740b5650 0x7fb2740b7b10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb290106d40 0x7fb2901a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb290102220 0x7fb29019fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 --2- 172.21.15.67:0/170724285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb290101900 0x7fb29019f550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.571+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 >> 172.21.15.67:0/170724285 conn(0x7fb2900fd5b0 msgr2=0x7fb290104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:27.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.572+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 shutdown_connections 2024-09-10T23:26:27.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:27.572+0000 7fb28f200640 1 -- 172.21.15.67:0/170724285 wait complete. 2024-09-10T23:26:28.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:27 smithi092 ceph-mon[34223]: pgmap v1255: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:26:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:27 smithi067 ceph-mon[30939]: pgmap v1255: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:26:28.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:27 smithi145 ceph-mon[34397]: pgmap v1255: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T23:26:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:29 smithi092 ceph-mon[34223]: pgmap v1256: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T23:26:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:29 smithi067 ceph-mon[30939]: pgmap v1256: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T23:26:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:29 smithi145 ceph-mon[34397]: pgmap v1256: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T23:26:32.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:31 smithi092 ceph-mon[34223]: pgmap v1257: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-09-10T23:26:32.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:31 smithi067 ceph-mon[30939]: pgmap v1257: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-09-10T23:26:32.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:31 smithi145 ceph-mon[34397]: pgmap v1257: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-09-10T23:26:34.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:33 smithi092 ceph-mon[34223]: pgmap v1258: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-10T23:26:34.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:33 smithi067 ceph-mon[30939]: pgmap v1258: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-10T23:26:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:33 smithi145 ceph-mon[34397]: pgmap v1258: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.4 MiB/s wr, 2.13k op/s 2024-09-10T23:26:35.689 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c91ff02b', quiesce took 34.2 seconds. Will release after: -4.200000000000003 2024-09-10T23:26:35.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:26:36.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:35 smithi092 ceph-mon[34223]: pgmap v1259: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-10T23:26:36.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:35 smithi067 ceph-mon[30939]: pgmap v1259: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-10T23:26:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:26:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:35 smithi145 ceph-mon[34397]: pgmap v1259: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.61k op/s 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 -- 172.21.15.67:0/1361712687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 msgr2=0x7f3624103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 --2- 172.21.15.67:0/1361712687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f3624103180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3614009900 tx=0x7f361402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 -- 172.21.15.67:0/1361712687 shutdown_connections 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 --2- 172.21.15.67:0/1361712687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 0x7f362410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 --2- 172.21.15.67:0/1361712687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 0x7f3624108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 --2- 172.21.15.67:0/1361712687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f3624103180 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.861+0000 7f362ae00640 1 -- 172.21.15.67:0/1361712687 >> 172.21.15.67:0/1361712687 conn(0x7f36240fe810 msgr2=0x7f3624100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.862+0000 7f362ae00640 1 -- 172.21.15.67:0/1361712687 shutdown_connections 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.862+0000 7f362ae00640 1 -- 172.21.15.67:0/1361712687 wait complete. 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.862+0000 7f362ae00640 1 Processor -- start 2024-09-10T23:26:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 -- start start 2024-09-10T23:26:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 0x7f362419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 0x7f36241a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3624112cb0 con 0x7f3624102da0 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3624112b30 con 0x7f3624109130 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f362ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3624112e30 con 0x7f36241087b0 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f3623e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f3623e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59698/0 (socket says 172.21.15.67:59698) 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f3623e00640 1 -- 172.21.15.67:0/2236918221 learned_addr learned my addr 172.21.15.67:0/2236918221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f3623400640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 0x7f362419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.863+0000 7f3628e00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 0x7f36241a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 -- 172.21.15.67:0/2236918221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 msgr2=0x7f362419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 0x7f362419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 -- 172.21.15.67:0/2236918221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 msgr2=0x7f36241a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 0x7f36241a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 -- 172.21.15.67:0/2236918221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241a13d0 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.864+0000 7f3623e00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f361402a910 tx=0x7f3614003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.865+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361403f070 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.865+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36241a1660 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.865+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3614003e40 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.865+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614035030 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.865+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36241a7d80 con 0x7f3624102da0 2024-09-10T23:26:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.866+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3624103180 con 0x7f3624102da0 2024-09-10T23:26:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.870+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3614031b90 con 0x7f3624102da0 2024-09-10T23:26:36.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.871+0000 7f3620c00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 0x7f36080b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:36.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.871+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9607+0+0 (secure 0 0 0) 0x7f36141073a0 con 0x7f3624102da0 2024-09-10T23:26:36.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.871+0000 7f3623400640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 0x7f36080b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:36.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.872+0000 7f3623400640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 0x7f36080b7910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3624101d30 tx=0x7f361000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:36.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:36.873+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f36140cda80 con 0x7f3624102da0 2024-09-10T23:26:37.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.115+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36241a8010 con 0x7f3624102da0 2024-09-10T23:26:37.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.115+0000 7f3620c00640 1 -- 172.21.15.67:0/2236918221 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11905 (secure 0 0 0) 0x7f36140cd380 con 0x7f3624102da0 2024-09-10T23:26:37.117 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:26:37.117 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"btime":"2024-09-10T23:25:31:756758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:25:31.060860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:26:37.117 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-09-10T23:26:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 msgr2=0x7f36080b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 0x7f36080b7910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3624101d30 tx=0x7f361000a2d0 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 msgr2=0x7f362419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f361402a910 tx=0x7f3614003c20 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 shutdown_connections 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f36080b5450 0x7f36080b7910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3624109130 0x7f36241a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36241087b0 0x7f362419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 --2- 172.21.15.67:0/2236918221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3624102da0 0x7f362419b320 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 >> 172.21.15.67:0/2236918221 conn(0x7f36240fe810 msgr2=0x7f36240ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 shutdown_connections 2024-09-10T23:26:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:37.118+0000 7f362ae00640 1 -- 172.21.15.67:0/2236918221 wait complete. 2024-09-10T23:26:37.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:37 smithi067 ceph-mon[30939]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.45k op/s 2024-09-10T23:26:37.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:37 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2236918221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:37 smithi092 ceph-mon[34223]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.45k op/s 2024-09-10T23:26:38.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:37 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2236918221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:38.086 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id c91ff02b --release --await' 2024-09-10T23:26:38.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id c91ff02b --release --await 2024-09-10T23:26:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:37 smithi145 ceph-mon[34397]: pgmap v1260: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 1.45k op/s 2024-09-10T23:26:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:37 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2236918221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:26:38.468 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 -- 172.21.15.67:0/2125694242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c10b330 msgr2=0x7f9d3c111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 --2- 172.21.15.67:0/2125694242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c10b330 0x7f9d3c111b30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009ca0 tx=0x7f9d3402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 -- 172.21.15.67:0/2125694242 shutdown_connections 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 --2- 172.21.15.67:0/2125694242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c10b330 0x7f9d3c111b30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 --2- 172.21.15.67:0/2125694242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 0x7f9d3c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 --2- 172.21.15.67:0/2125694242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 -- 172.21.15.67:0/2125694242 >> 172.21.15.67:0/2125694242 conn(0x7f9d3c100a50 msgr2=0x7f9d3c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 -- 172.21.15.67:0/2125694242 shutdown_connections 2024-09-10T23:26:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.200+0000 7f9d43a00640 1 -- 172.21.15.67:0/2125694242 wait complete. 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.201+0000 7f9d43a00640 1 Processor -- start 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 -- start start 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 0x7f9d3c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c10b330 0x7f9d3c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c114e60 con 0x7f9d3c104fa0 2024-09-10T23:26:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c114ce0 con 0x7f9d3c10b330 2024-09-10T23:26:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d43a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c114fe0 con 0x7f9d3c10a9b0 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38866/0 (socket says 172.21.15.67:38866) 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.202+0000 7f9d41000640 1 -- 172.21.15.67:0/3080680762 learned_addr learned my addr 172.21.15.67:0/3080680762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c10b330 0x7f9d3c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d33e00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 0x7f9d3c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 -- 172.21.15.67:0/3080680762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 msgr2=0x7f9d3c19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 0x7f9d3c19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 -- 172.21.15.67:0/3080680762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c10b330 msgr2=0x7f9d3c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c10b330 0x7f9d3c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 -- 172.21.15.67:0/3080680762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c1a1420 con 0x7f9d3c104fa0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.203+0000 7f9d41000640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c890 tx=0x7f9d2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.204+0000 7f9d3a000640 1 -- 172.21.15.67:0/3080680762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004220 con 0x7f9d3c104fa0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.204+0000 7f9d3a000640 1 -- 172.21.15.67:0/3080680762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d2c007e40 con 0x7f9d3c104fa0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.204+0000 7f9d3a000640 1 -- 172.21.15.67:0/3080680762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004a00 con 0x7f9d3c104fa0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.204+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c1a1710 con 0x7f9d3c104fa0 2024-09-10T23:26:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.204+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c1a7fa0 con 0x7f9d3c104fa0 2024-09-10T23:26:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.207+0000 7f9d3a000640 1 -- 172.21.15.67:0/3080680762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9d2c0079b0 con 0x7f9d3c104fa0 2024-09-10T23:26:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.208+0000 7f9d3a000640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 0x7f9d200b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.209+0000 7f9d3a000640 1 -- 172.21.15.67:0/3080680762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9519+0+0 (secure 0 0 0) 0x7f9d2c0e5fe0 con 0x7f9d3c104fa0 2024-09-10T23:26:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.209+0000 7f9d33e00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 0x7f9d200b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.209+0000 7f9d33e00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 0x7f9d200b7990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c107380 tx=0x7f9d2400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.214+0000 7f9d4661f740 1 Processor -- start 2024-09-10T23:26:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 -- start start 2024-09-10T23:26:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 0x5596ebe6d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe79360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebe7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5596ebde4290 con 0x5596ebe798a0 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5596ebdd36c0 con 0x5596ebe6d910 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d4661f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5596ebdd74d0 con 0x5596ebe6cff0 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d41a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebe7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d41a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebe7bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38882/0 (socket says 172.21.15.67:38882) 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d41a00640 1 -- 172.21.15.67:0/2195994592 learned_addr learned my addr 172.21.15.67:0/2195994592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d33e00640 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe79360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.215+0000 7f9d41000640 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 0x5596ebe6d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 -- 172.21.15.67:0/2195994592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 msgr2=0x5596ebe79360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe79360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 -- 172.21.15.67:0/2195994592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 msgr2=0x5596ebe7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebe7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 -- 172.21.15.67:0/2195994592 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ebd60460 con 0x5596ebe6cff0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d41000640 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 0x5596ebe6d3d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0e3310 tx=0x7f9d2c0a80a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f809e852c4c3718e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d32000640 1 -- 172.21.15.67:0/2195994592 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01d040 con 0x5596ebe6cff0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d32000640 1 -- 172.21.15.67:0/2195994592 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d2c0ac0f0 con 0x5596ebe6cff0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.216+0000 7f9d32000640 1 -- 172.21.15.67:0/2195994592 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0a7a40 con 0x5596ebe6cff0 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 -- 172.21.15.67:0/2195994592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 msgr2=0x5596ebe6d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 0x5596ebe6d3d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0e3310 tx=0x7f9d2c0a80a0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 -- 172.21.15.67:0/2195994592 shutdown_connections 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebe7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe79360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 --2- 172.21.15.67:0/2195994592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6cff0 0x5596ebe6d3d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 -- 172.21.15.67:0/2195994592 >> 172.21.15.67:0/2195994592 conn(0x5596ebe6ae20 msgr2=0x5596ebe6bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 -- 172.21.15.67:0/2195994592 shutdown_connections 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.217+0000 7f9d4661f740 1 -- 172.21.15.67:0/2195994592 wait complete. 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 -- start start 2024-09-10T23:26:39.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 Processor -- start 2024-09-10T23:26:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe798a0 0x5596ebe76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5596ebc34bb0 con 0x5596ebe6cff0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5596ebd6e6c0 con 0x5596ebe798a0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5596ebe7c550 con 0x5596ebe6d910 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.218+0000 7f9d4661f740 10 client.0 _ll_register_callbacks cb 0x5596ebd89490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38892/0 (socket says 172.21.15.67:38892) 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 -- 172.21.15.67:0/4160692202 learned_addr learned my addr 172.21.15.67:0/4160692202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d33e00640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41a00640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe798a0 0x5596ebe76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 -- 172.21.15.67:0/4160692202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 msgr2=0x5596ebe76740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe76740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 -- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe798a0 msgr2=0x5596ebe76ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe798a0 0x5596ebe76ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 -- 172.21.15.67:0/4160692202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ebb31b00 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d41000640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0e6280 tx=0x7f9d2c0ab580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb5562a75865f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.219+0000 7f9d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01d040 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9519+0+0 (secure 0 0 0) 0x7f9d2c0a7c50 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2c0ac0f0 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c0ab670 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d4661f740 10 client.16587 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5596ebba8960 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d2c0ac0f0 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f9d2c0b1db0 con 0x5596ebe6cff0 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d4661f740 10 client.16587 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:39.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.220+0000 7f9d4661f740 10 client.16587 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.221+0000 7f9d4661f740 20 client.16587 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.221+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebe891e0 0x5596ebe8b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.221+0000 7f9d4661f740 4 client.16587 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:26:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.221+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596ebde61e0 con 0x5596ebe891e0 2024-09-10T23:26:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.221+0000 7f9d33e00640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebe891e0 0x5596ebe8b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.222+0000 7f9d33e00640 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebe891e0 0x5596ebe8b5d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67422800b3c30ee server_cookie=571a054afe3c489f in_seq=0 out_seq=0 2024-09-10T23:26:39.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.222+0000 7f9d2aa00640 10 client.16587 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:39.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d2aa00640 1 -- 172.21.15.67:0/4160692202 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596ebde61e0 con 0x5596ebe891e0 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d2aa00640 10 client.16587 handle_command_reply: tid=0 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 client.16587 shutdown 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 msgr2=0x5596ebe792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0e6280 tx=0x7f9d2c0ab580 comp rx=0 tx=0).stop 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 shutdown_connections 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebe891e0 0x5596ebe8b5d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe798a0 0x5596ebe76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 --2- 172.21.15.67:0/4160692202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe792f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d2aa00640 0 client.16587 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.223+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 >> 172.21.15.67:0/4160692202 conn(0x5596ebe6ae20 msgr2=0x5596ebe709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.224+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 shutdown_connections 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.224+0000 7f9d4661f740 1 -- 172.21.15.67:0/4160692202 wait complete. 2024-09-10T23:26:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.224+0000 7f9d4661f740 20 client.16587 trim_cache size 0 max 16384 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 Processor -- start 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 -- start start 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 0x5596ebe73950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b4a40 con 0x5596ebe6cff0 2024-09-10T23:26:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b22b0 con 0x5596ebe6d910 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.243+0000 7f9d4661f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b0120 con 0x5596ebe798a0 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38902/0 (socket says 172.21.15.67:38902) 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 -- 172.21.15.67:0/1100569133 learned_addr learned my addr 172.21.15.67:0/1100569133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d33e00640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41a00640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 0x5596ebe73950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 -- 172.21.15.67:0/1100569133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 msgr2=0x5596ebe73950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 0x5596ebe73950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 -- 172.21.15.67:0/1100569133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 msgr2=0x5596ebe69c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe69c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 -- 172.21.15.67:0/1100569133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ebe6c500 con 0x5596ebe6cff0 2024-09-10T23:26:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41a00640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 0x5596ebe73950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.244+0000 7f9d41000640 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a8670 tx=0x7f9d2c0e3e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca72bdad3c3dbb28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d31600640 1 -- 172.21.15.67:0/1100569133 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0ab5f0 con 0x5596ebe6cff0 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d31600640 1 -- 172.21.15.67:0/1100569133 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d2c0a7c50 con 0x5596ebe6cff0 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d31600640 1 -- 172.21.15.67:0/1100569133 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0b1f10 con 0x5596ebe6cff0 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 -- 172.21.15.67:0/1100569133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 msgr2=0x5596ebe71bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a8670 tx=0x7f9d2c0e3e00 comp rx=0 tx=0).stop 2024-09-10T23:26:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 -- 172.21.15.67:0/1100569133 shutdown_connections 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe798a0 0x5596ebe73950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6d910 0x5596ebe69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 --2- 172.21.15.67:0/1100569133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe6cff0 0x5596ebe71bc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 -- 172.21.15.67:0/1100569133 >> 172.21.15.67:0/1100569133 conn(0x5596ebe6ae20 msgr2=0x5596ebe78590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.245+0000 7f9d4661f740 1 -- 172.21.15.67:0/1100569133 shutdown_connections 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.246+0000 7f9d4661f740 1 -- 172.21.15.67:0/1100569133 wait complete. 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.246+0000 7f9d4661f740 1 -- start start 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.246+0000 7f9d4661f740 1 Processor -- start 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.246+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6cff0 0x5596ebe8bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe8c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b0770 con 0x5596ebe798a0 2024-09-10T23:26:39.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b1be0 con 0x5596ebe6cff0 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b0e40 con 0x5596ebe6d910 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d4661f740 10 client.0 _ll_register_callbacks cb 0x7f9d2c0aba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38906/0 (socket says 172.21.15.67:38906) 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 -- 172.21.15.67:0/3923802094 learned_addr learned my addr 172.21.15.67:0/3923802094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d33e00640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe8c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41000640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6cff0 0x5596ebe8bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 -- 172.21.15.67:0/3923802094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 msgr2=0x5596ebe8c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe8c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 -- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6cff0 msgr2=0x5596ebe8bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6cff0 0x5596ebe8bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.247+0000 7f9d41a00640 1 -- 172.21.15.67:0/3923802094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ebef04a0 con 0x5596ebe798a0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d41a00640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009ca0 tx=0x7f9d34031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d093e783fa4282e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34031e60 con 0x5596ebe798a0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9519+0+0 (secure 0 0 0) 0x7f9d34003d80 con 0x5596ebe798a0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2c0a7c50 con 0x5596ebe798a0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9d34054050 con 0x5596ebe798a0 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d4661f740 10 client.16593 fetch_fsmap learned FSMap version 41 2024-09-10T23:26:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.248+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5596ebef78a0 con 0x5596ebe798a0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d2c0a7c50 con 0x5596ebe798a0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f9d3404f030 con 0x5596ebe798a0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 10 client.16593 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 10 client.16593 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 20 client.16593 populate_metadata read hostname 'smithi067' 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebef80a0 0x5596ebefa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 4 client.16593 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c91ff02b", "release": true, "await": true}] 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c91ff02b", "release": true, "await": true}) -- 0x5596ebde1c90 con 0x5596ebef80a0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.249+0000 7f9d41000640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebef80a0 0x5596ebefa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.250+0000 7f9d41000640 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebef80a0 0x5596ebefa4c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75adb7f4695b0b21 server_cookie=fc79bb09c5ea0401 in_seq=0 out_seq=0 2024-09-10T23:26:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.250+0000 7f9d2aa00640 10 client.16593 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:39.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.281+0000 7f9d2aa00640 1 -- 172.21.15.67:0/3923802094 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5596ebde1c90 con 0x5596ebef80a0 2024-09-10T23:26:39.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.281+0000 7f9d2aa00640 10 client.16593 handle_command_reply: tid=0 2024-09-10T23:26:39.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.281+0000 7f9d4661f740 1 client.16593 shutdown 2024-09-10T23:26:39.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.281+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 msgr2=0x5596ebeefd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.281+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009ca0 tx=0x7f9d34031cf0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 shutdown_connections 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5596ebef80a0 0x5596ebefa4c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5596ebe798a0 0x5596ebeefd90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596ebe6d910 0x5596ebe8c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 --2- 172.21.15.67:0/3923802094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5596ebe6cff0 0x5596ebe8bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.282+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 >> 172.21.15.67:0/3923802094 conn(0x5596ebe6ae20 msgr2=0x5596ebe74fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.284+0000 7f9d2aa00640 0 client.16593 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:26:39.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.284+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 shutdown_connections 2024-09-10T23:26:39.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.285+0000 7f9d4661f740 1 -- 172.21.15.67:0/3923802094 wait complete. 2024-09-10T23:26:39.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.285+0000 7f9d4661f740 20 client.16593 trim_cache size 0 max 16384 2024-09-10T23:26:39.287 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:26:39.287 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 41, 2024-09-10T23:26:39.287 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:26:39.287 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 84, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "c91ff02b": { 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 84, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 150.0, 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:26:39.288 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RELEASED", 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:26:39.289 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:26:39.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.288+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 msgr2=0x7f9d200b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.288+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 0x7f9d200b7990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c107380 tx=0x7f9d2400b390 comp rx=0 tx=0).stop 2024-09-10T23:26:39.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.288+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 msgr2=0x7f9d3c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.288+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c890 tx=0x7f9d2c00cd50 comp rx=0 tx=0).stop 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 shutdown_connections 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9d200b54d0 0x7f9d200b7990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d3c10b330 0x7f9d3c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d3c10a9b0 0x7f9d3c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 --2- 172.21.15.67:0/3080680762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d3c104fa0 0x7f9d3c19b370 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 >> 172.21.15.67:0/3080680762 conn(0x7f9d3c100a50 msgr2=0x7f9d3c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 shutdown_connections 2024-09-10T23:26:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:26:39.289+0000 7f9d43a00640 1 -- 172.21.15.67:0/3080680762 wait complete. 2024-09-10T23:26:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:39 smithi067 ceph-mon[30939]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 532 op/s 2024-09-10T23:26:40.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:39 smithi067 ceph-mon[30939]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T23:26:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:39 smithi145 ceph-mon[34397]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 532 op/s 2024-09-10T23:26:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:39 smithi145 ceph-mon[34397]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T23:26:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:39 smithi092 ceph-mon[34223]: pgmap v1261: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.1 MiB/s wr, 532 op/s 2024-09-10T23:26:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:39 smithi092 ceph-mon[34223]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T23:26:40.313 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c91ff02b', total seconds elapsed: 38.8 2024-09-10T23:26:41.780 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:41 smithi067 ceph-mon[30939]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 942 KiB/s wr, 233 op/s 2024-09-10T23:26:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:41 smithi145 ceph-mon[34397]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 942 KiB/s wr, 233 op/s 2024-09-10T23:26:42.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:41 smithi092 ceph-mon[34223]: pgmap v1263: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 942 KiB/s wr, 233 op/s 2024-09-10T23:26:44.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:43 smithi067 ceph-mon[30939]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-10T23:26:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:43 smithi145 ceph-mon[34397]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-10T23:26:44.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:43 smithi092 ceph-mon[34223]: pgmap v1264: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-10T23:26:46.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:45 smithi067 ceph-mon[30939]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.1 MiB/s wr, 537 op/s 2024-09-10T23:26:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:45 smithi145 ceph-mon[34397]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.1 MiB/s wr, 537 op/s 2024-09-10T23:26:46.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:45 smithi092 ceph-mon[34223]: pgmap v1265: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.1 MiB/s wr, 537 op/s 2024-09-10T23:26:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:47 smithi145 ceph-mon[34397]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 1.72k op/s 2024-09-10T23:26:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:47 smithi092 ceph-mon[34223]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 1.72k op/s 2024-09-10T23:26:48.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:47 smithi067 ceph-mon[30939]: pgmap v1266: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 1.72k op/s 2024-09-10T23:26:50.105 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:49 smithi067 ceph-mon[30939]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 1.72k op/s 2024-09-10T23:26:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:49 smithi145 ceph-mon[34397]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 1.72k op/s 2024-09-10T23:26:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:49 smithi092 ceph-mon[34223]: pgmap v1267: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 1.72k op/s 2024-09-10T23:26:51.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:50 smithi145 ceph-mon[34397]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 1.88k op/s 2024-09-10T23:26:51.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:50 smithi092 ceph-mon[34223]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 1.88k op/s 2024-09-10T23:26:51.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:50 smithi067 ceph-mon[30939]: pgmap v1268: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 1.88k op/s 2024-09-10T23:26:54.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:53 smithi092 ceph-mon[34223]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-09-10T23:26:54.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:53 smithi067 ceph-mon[30939]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-09-10T23:26:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:53 smithi145 ceph-mon[34397]: pgmap v1269: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-09-10T23:26:56.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:55 smithi092 ceph-mon[34223]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:26:56.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:55 smithi067 ceph-mon[30939]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:26:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:55 smithi145 ceph-mon[34397]: pgmap v1270: 129 pgs: 129 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T23:26:58.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:57 smithi092 ceph-mon[34223]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T23:26:58.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:57 smithi067 ceph-mon[30939]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T23:26:58.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:57 smithi145 ceph-mon[34397]: pgmap v1271: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T23:26:59.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:58 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:26:59.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:58 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:26:59.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:58 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:27:00.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:26:59 smithi067 ceph-mon[30939]: pgmap v1272: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:26:59 smithi145 ceph-mon[34397]: pgmap v1272: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:26:59 smithi092 ceph-mon[34223]: pgmap v1272: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:02.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:01 smithi067 ceph-mon[30939]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-10T23:27:02.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:01 smithi092 ceph-mon[34223]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-10T23:27:02.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:01 smithi145 ceph-mon[34397]: pgmap v1273: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-10T23:27:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:03 smithi145 ceph-mon[34397]: pgmap v1274: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:27:04.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:03 smithi092 ceph-mon[34223]: pgmap v1274: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:27:04.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:03 smithi067 ceph-mon[30939]: pgmap v1274: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T23:27:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:05 smithi145 ceph-mon[34397]: pgmap v1275: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:05 smithi092 ceph-mon[34223]: pgmap v1275: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:06.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:05 smithi067 ceph-mon[30939]: pgmap v1275: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 2.32k op/s 2024-09-10T23:27:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:07 smithi145 ceph-mon[34397]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T23:27:08.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:07 smithi092 ceph-mon[34223]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T23:27:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:07 smithi067 ceph-mon[30939]: pgmap v1276: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T23:27:09.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:08 smithi145 ceph-mon[34397]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-09-10T23:27:09.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:08 smithi092 ceph-mon[34223]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-09-10T23:27:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:08 smithi067 ceph-mon[30939]: pgmap v1277: 129 pgs: 129 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-09-10T23:27:12.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:11 smithi092 ceph-mon[34223]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:27:12.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:11 smithi067 ceph-mon[30939]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:27:12.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:11 smithi145 ceph-mon[34397]: pgmap v1278: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-09-10T23:27:14.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:13 smithi092 ceph-mon[34223]: pgmap v1279: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T23:27:14.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:13 smithi067 ceph-mon[30939]: pgmap v1279: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T23:27:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:13 smithi145 ceph-mon[34397]: pgmap v1279: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T23:27:16.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:15 smithi092 ceph-mon[34223]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:27:16.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:15 smithi067 ceph-mon[30939]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:27:16.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:15 smithi145 ceph-mon[34397]: pgmap v1280: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T23:27:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:17 smithi092 ceph-mon[34223]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:27:18.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:17 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:27:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:17 smithi067 ceph-mon[30939]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:27:18.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:17 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:27:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:17 smithi145 ceph-mon[34397]: pgmap v1281: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-10T23:27:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:17 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:27:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:19 smithi067 ceph-mon[30939]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 1.74k op/s 2024-09-10T23:27:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:19 smithi067 ceph-mon[30939]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T23:27:20.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:19 smithi145 ceph-mon[34397]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 1.74k op/s 2024-09-10T23:27:20.132 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:19 smithi145 ceph-mon[34397]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T23:27:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:19 smithi092 ceph-mon[34223]: pgmap v1282: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.9 MiB/s wr, 1.74k op/s 2024-09-10T23:27:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:19 smithi092 ceph-mon[34223]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T23:27:21.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:21 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:27:21.383 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:21 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:27:21.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:21 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:27:22.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:22 smithi067 ceph-mon[30939]: pgmap v1284: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:27:22.382 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:22 smithi145 ceph-mon[34397]: pgmap v1284: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:27:22.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:22 smithi092 ceph-mon[34223]: pgmap v1284: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 1.33k op/s 2024-09-10T23:27:23.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:23 smithi145 ceph-mon[34397]: pgmap v1285: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.6 MiB/s wr, 650 op/s 2024-09-10T23:27:23.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:23 smithi092 ceph-mon[34223]: pgmap v1285: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.6 MiB/s wr, 650 op/s 2024-09-10T23:27:23.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:23 smithi067 ceph-mon[30939]: pgmap v1285: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.6 MiB/s wr, 650 op/s 2024-09-10T23:27:24.318 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:27:24.318 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:27:24.318 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:27:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:27:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:27:24.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:27:24.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:27:24.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:27:24.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:27:25.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:25 smithi145 ceph-mon[34397]: pgmap v1286: 129 pgs: 13 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 84 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-10T23:27:25.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:25 smithi092 ceph-mon[34223]: pgmap v1286: 129 pgs: 13 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 84 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-10T23:27:25.547 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:25 smithi067 ceph-mon[30939]: pgmap v1286: 129 pgs: 13 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 84 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-10T23:27:28.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:27 smithi092 ceph-mon[34223]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 632 KiB/s wr, 160 op/s 2024-09-10T23:27:28.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:27 smithi067 ceph-mon[30939]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 632 KiB/s wr, 160 op/s 2024-09-10T23:27:28.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:27 smithi145 ceph-mon[34397]: pgmap v1287: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 632 KiB/s wr, 160 op/s 2024-09-10T23:27:30.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:29 smithi092 ceph-mon[34223]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 522 KiB/s wr, 134 op/s 2024-09-10T23:27:30.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:29 smithi092 ceph-mon[34223]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T23:27:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:29 smithi067 ceph-mon[30939]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 522 KiB/s wr, 134 op/s 2024-09-10T23:27:30.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:29 smithi067 ceph-mon[30939]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T23:27:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:29 smithi145 ceph-mon[34397]: pgmap v1288: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 522 KiB/s wr, 134 op/s 2024-09-10T23:27:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:29 smithi145 ceph-mon[34397]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T23:27:32.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:31 smithi092 ceph-mon[34223]: pgmap v1290: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 598 KiB/s wr, 153 op/s 2024-09-10T23:27:32.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:31 smithi067 ceph-mon[30939]: pgmap v1290: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 598 KiB/s wr, 153 op/s 2024-09-10T23:27:32.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:31 smithi145 ceph-mon[34397]: pgmap v1290: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 598 KiB/s wr, 153 op/s 2024-09-10T23:27:34.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:33 smithi092 ceph-mon[34223]: pgmap v1291: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 822 KiB/s wr, 211 op/s 2024-09-10T23:27:34.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:33 smithi067 ceph-mon[30939]: pgmap v1291: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 822 KiB/s wr, 211 op/s 2024-09-10T23:27:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:33 smithi145 ceph-mon[34397]: pgmap v1291: 129 pgs: 129 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 822 KiB/s wr, 211 op/s 2024-09-10T23:27:36.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:35 smithi092 ceph-mon[34223]: pgmap v1292: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 836 KiB/s wr, 215 op/s 2024-09-10T23:27:36.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:35 smithi067 ceph-mon[30939]: pgmap v1292: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 836 KiB/s wr, 215 op/s 2024-09-10T23:27:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:35 smithi145 ceph-mon[34397]: pgmap v1292: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 836 KiB/s wr, 215 op/s 2024-09-10T23:27:38.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:37 smithi092 ceph-mon[34223]: pgmap v1293: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1008 KiB/s wr, 257 op/s 2024-09-10T23:27:38.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:37 smithi067 ceph-mon[30939]: pgmap v1293: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1008 KiB/s wr, 257 op/s 2024-09-10T23:27:38.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:37 smithi145 ceph-mon[34397]: pgmap v1293: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1008 KiB/s wr, 257 op/s 2024-09-10T23:27:40.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:39 smithi067 ceph-mon[30939]: pgmap v1294: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1021 KiB/s wr, 261 op/s 2024-09-10T23:27:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:39 smithi145 ceph-mon[34397]: pgmap v1294: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1021 KiB/s wr, 261 op/s 2024-09-10T23:27:40.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:39 smithi092 ceph-mon[34223]: pgmap v1294: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1021 KiB/s wr, 261 op/s 2024-09-10T23:27:42.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:41 smithi067 ceph-mon[30939]: pgmap v1295: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 KiB/s wr, 240 op/s 2024-09-10T23:27:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:41 smithi145 ceph-mon[34397]: pgmap v1295: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 KiB/s wr, 240 op/s 2024-09-10T23:27:42.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:41 smithi092 ceph-mon[34223]: pgmap v1295: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 KiB/s wr, 240 op/s 2024-09-10T23:27:44.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:43 smithi067 ceph-mon[30939]: pgmap v1296: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.0 MiB/s wr, 268 op/s 2024-09-10T23:27:44.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:43 smithi145 ceph-mon[34397]: pgmap v1296: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.0 MiB/s wr, 268 op/s 2024-09-10T23:27:44.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:43 smithi092 ceph-mon[34223]: pgmap v1296: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.0 MiB/s wr, 268 op/s 2024-09-10T23:27:46.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:45 smithi067 ceph-mon[30939]: pgmap v1297: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 691 KiB/s wr, 177 op/s 2024-09-10T23:27:46.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:45 smithi145 ceph-mon[34397]: pgmap v1297: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 691 KiB/s wr, 177 op/s 2024-09-10T23:27:46.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:45 smithi092 ceph-mon[34223]: pgmap v1297: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 691 KiB/s wr, 177 op/s 2024-09-10T23:27:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:47 smithi067 ceph-mon[30939]: pgmap v1298: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 917 KiB/s wr, 236 op/s 2024-09-10T23:27:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:47 smithi145 ceph-mon[34397]: pgmap v1298: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 917 KiB/s wr, 236 op/s 2024-09-10T23:27:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:47 smithi092 ceph-mon[34223]: pgmap v1298: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 917 KiB/s wr, 236 op/s 2024-09-10T23:27:50.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:49 smithi067 ceph-mon[30939]: pgmap v1299: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 574 KiB/s wr, 149 op/s 2024-09-10T23:27:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:49 smithi145 ceph-mon[34397]: pgmap v1299: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 574 KiB/s wr, 149 op/s 2024-09-10T23:27:50.413 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:49 smithi092 ceph-mon[34223]: pgmap v1299: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 574 KiB/s wr, 149 op/s 2024-09-10T23:27:52.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:51 smithi067 ceph-mon[30939]: pgmap v1300: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 599 KiB/s wr, 156 op/s 2024-09-10T23:27:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:51 smithi145 ceph-mon[34397]: pgmap v1300: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 599 KiB/s wr, 156 op/s 2024-09-10T23:27:52.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:51 smithi092 ceph-mon[34223]: pgmap v1300: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 599 KiB/s wr, 156 op/s 2024-09-10T23:27:54.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:53 smithi067 ceph-mon[30939]: pgmap v1301: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 762 KiB/s wr, 197 op/s 2024-09-10T23:27:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:53 smithi145 ceph-mon[34397]: pgmap v1301: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 762 KiB/s wr, 197 op/s 2024-09-10T23:27:54.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:53 smithi092 ceph-mon[34223]: pgmap v1301: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 762 KiB/s wr, 197 op/s 2024-09-10T23:27:56.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:55 smithi067 ceph-mon[30939]: pgmap v1302: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 550 KiB/s wr, 142 op/s 2024-09-10T23:27:56.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:55 smithi145 ceph-mon[34397]: pgmap v1302: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 550 KiB/s wr, 142 op/s 2024-09-10T23:27:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:55 smithi092 ceph-mon[34223]: pgmap v1302: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 550 KiB/s wr, 142 op/s 2024-09-10T23:27:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:57 smithi145 ceph-mon[34397]: pgmap v1303: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 777 KiB/s wr, 201 op/s 2024-09-10T23:27:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:57 smithi092 ceph-mon[34223]: pgmap v1303: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 777 KiB/s wr, 201 op/s 2024-09-10T23:27:58.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:57 smithi067 ceph-mon[30939]: pgmap v1303: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 777 KiB/s wr, 201 op/s 2024-09-10T23:28:00.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:27:59 smithi145 ceph-mon[34397]: pgmap v1304: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 559 KiB/s wr, 145 op/s 2024-09-10T23:28:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:27:59 smithi092 ceph-mon[34223]: pgmap v1304: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 559 KiB/s wr, 145 op/s 2024-09-10T23:28:00.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:27:59 smithi067 ceph-mon[30939]: pgmap v1304: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 559 KiB/s wr, 145 op/s 2024-09-10T23:28:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:01 smithi145 ceph-mon[34397]: pgmap v1305: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 599 KiB/s wr, 155 op/s 2024-09-10T23:28:02.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:01 smithi092 ceph-mon[34223]: pgmap v1305: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 599 KiB/s wr, 155 op/s 2024-09-10T23:28:02.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:01 smithi067 ceph-mon[30939]: pgmap v1305: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 599 KiB/s wr, 155 op/s 2024-09-10T23:28:04.073 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:03 smithi092 ceph-mon[34223]: pgmap v1306: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 809 KiB/s wr, 212 op/s 2024-09-10T23:28:04.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:03 smithi145 ceph-mon[34397]: pgmap v1306: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 809 KiB/s wr, 212 op/s 2024-09-10T23:28:04.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:03 smithi067 ceph-mon[30939]: pgmap v1306: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 809 KiB/s wr, 212 op/s 2024-09-10T23:28:06.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:05 smithi145 ceph-mon[34397]: pgmap v1307: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 596 KiB/s wr, 158 op/s 2024-09-10T23:28:06.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:05 smithi092 ceph-mon[34223]: pgmap v1307: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 596 KiB/s wr, 158 op/s 2024-09-10T23:28:06.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:05 smithi067 ceph-mon[30939]: pgmap v1307: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 596 KiB/s wr, 158 op/s 2024-09-10T23:28:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:07 smithi145 ceph-mon[34397]: pgmap v1308: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 811 KiB/s wr, 214 op/s 2024-09-10T23:28:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:07 smithi092 ceph-mon[34223]: pgmap v1308: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 811 KiB/s wr, 214 op/s 2024-09-10T23:28:08.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:07 smithi067 ceph-mon[30939]: pgmap v1308: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 811 KiB/s wr, 214 op/s 2024-09-10T23:28:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:08 smithi145 ceph-mon[34397]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T23:28:09.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:08 smithi092 ceph-mon[34223]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T23:28:09.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:08 smithi067 ceph-mon[30939]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T23:28:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:09 smithi145 ceph-mon[34397]: pgmap v1310: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 183 op/s 2024-09-10T23:28:10.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:09 smithi092 ceph-mon[34223]: pgmap v1310: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 183 op/s 2024-09-10T23:28:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:09 smithi067 ceph-mon[30939]: pgmap v1310: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 692 KiB/s wr, 183 op/s 2024-09-10T23:28:11.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:10 smithi092 ceph-mon[34223]: pgmap v1311: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 657 KiB/s wr, 175 op/s 2024-09-10T23:28:11.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:10 smithi067 ceph-mon[30939]: pgmap v1311: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 657 KiB/s wr, 175 op/s 2024-09-10T23:28:11.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:10 smithi145 ceph-mon[34397]: pgmap v1311: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 657 KiB/s wr, 175 op/s 2024-09-10T23:28:14.020 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:13 smithi092 ceph-mon[34223]: pgmap v1312: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 436 KiB/s wr, 115 op/s 2024-09-10T23:28:14.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:13 smithi067 ceph-mon[30939]: pgmap v1312: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 436 KiB/s wr, 115 op/s 2024-09-10T23:28:14.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:13 smithi145 ceph-mon[34397]: pgmap v1312: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 436 KiB/s wr, 115 op/s 2024-09-10T23:28:16.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:15 smithi092 ceph-mon[34223]: pgmap v1313: 129 pgs: 14 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 76 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 429 KiB/s wr, 113 op/s 2024-09-10T23:28:16.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:15 smithi067 ceph-mon[30939]: pgmap v1313: 129 pgs: 14 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 76 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 429 KiB/s wr, 113 op/s 2024-09-10T23:28:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:15 smithi145 ceph-mon[34397]: pgmap v1313: 129 pgs: 14 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 76 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 429 KiB/s wr, 113 op/s 2024-09-10T23:28:18.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:17 smithi067 ceph-mon[30939]: pgmap v1314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-09-10T23:28:18.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:17 smithi145 ceph-mon[34397]: pgmap v1314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-09-10T23:28:18.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:17 smithi092 ceph-mon[34223]: pgmap v1314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-09-10T23:28:20.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:19 smithi067 ceph-mon[30939]: pgmap v1315: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 KiB/s wr, 83 op/s 2024-09-10T23:28:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:19 smithi067 ceph-mon[30939]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T23:28:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:19 smithi145 ceph-mon[34397]: pgmap v1315: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 KiB/s wr, 83 op/s 2024-09-10T23:28:20.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:19 smithi145 ceph-mon[34397]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T23:28:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:19 smithi092 ceph-mon[34223]: pgmap v1315: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 KiB/s wr, 83 op/s 2024-09-10T23:28:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:19 smithi092 ceph-mon[34223]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T23:28:22.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:21 smithi067 ceph-mon[30939]: pgmap v1317: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 359 KiB/s wr, 93 op/s 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:Syncing()...3 sec 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:FFSB benchmark finished at: Tue Sep 10 23:28:22 2024 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:Results: 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:Benchmark took 383.97 sec 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:Total Results 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout:=============== 2024-09-10T23:28:22.081 INFO:tasks.workunit.client.0.smithi067.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout: ======= ============ ========= ======= =========== ========== 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout: write : 821760 2140.19 100.000% 100.000% 8.36MB/sec 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout:- 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout:2140.19 Transactions per Second 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout:Throughput Results 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout:=================== 2024-09-10T23:28:22.082 INFO:tasks.workunit.client.0.smithi067.stdout:Write Throughput: 8.36MB/sec 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout:System Call Latency statistics in millisecs 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout:===== 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout: Min Avg Max Total Calls 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout: ======== ======== ======== ============ 2024-09-10T23:28:22.090 INFO:tasks.workunit.client.0.smithi067.stdout:[ open] 0.868000 226.059846 12784.722656 642 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[0] 5000.000000 - 10000.000000 : 12 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[1] 2000.000000 - 5000.000000 : 4 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[2] 1000.000000 - 2000.000000 : 6 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[3] 500.000000 - 1000.000000 : 5 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[4] 200.000000 - 500.000000 : 5 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[5] 100.000000 - 200.000000 : 7 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[6] 50.000000 - 100.000000 : 73 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[7] 20.000000 - 50.000000 : 357 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[8] 10.000000 - 20.000000 : 131 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[9] 5.000000 - 10.000000 : 26 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[10] 2.000000 - 5.000000 : 12 2024-09-10T23:28:22.091 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[11] 1.000000 - 2.000000 : 1 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout:[ write] 0.173000 12.325988 57304.394531 821760 2024-09-10T23:28:22.092 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[0] 5000.000000 - 10000.000000 : 44 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[1] 2000.000000 - 5000.000000 : 62 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[2] 1000.000000 - 2000.000000 : 70 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[3] 500.000000 - 1000.000000 : 124 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[4] 200.000000 - 500.000000 : 641 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[5] 100.000000 - 200.000000 : 746 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[6] 50.000000 - 100.000000 : 5318 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[7] 20.000000 - 50.000000 : 54427 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[8] 10.000000 - 20.000000 : 145591 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[9] 5.000000 - 10.000000 : 210883 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[10] 2.000000 - 5.000000 : 185979 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[11] 1.000000 - 2.000000 : 60880 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[12] 0.500000 - 1.000000 : 78416 2024-09-10T23:28:22.093 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[13] 0.200000 - 0.500000 : 78035 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[14] 0.100000 - 0.200000 : 389 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout:[ lseek] 0.000000 0.002079 7.433000 821760 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-10T23:28:22.094 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[9] 5.000000 - 10.000000 : 3 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[10] 2.000000 - 5.000000 : 17 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[11] 1.000000 - 2.000000 : 33 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[12] 0.500000 - 1.000000 : 61 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[13] 0.200000 - 0.500000 : 185 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[14] 0.100000 - 0.200000 : 149 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[15] 0.050000 - 0.100000 : 168 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[16] 0.020000 - 0.050000 : 978 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[17] 0.010000 - 0.020000 : 1177 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[18] 0.000000 - 0.010000 : 818989 2024-09-10T23:28:22.095 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout:[ close] 0.037000 3.617693 49.098999 642 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[7] 20.000000 - 50.000000 : 28 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[8] 10.000000 - 20.000000 : 28 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[9] 5.000000 - 10.000000 : 84 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[10] 2.000000 - 5.000000 : 108 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[11] 1.000000 - 2.000000 : 54 2024-09-10T23:28:22.096 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[12] 0.500000 - 1.000000 : 35 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[13] 0.200000 - 0.500000 : 87 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[14] 0.100000 - 0.200000 : 77 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[15] 0.050000 - 0.100000 : 137 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[16] 0.020000 - 0.050000 : 4 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout:0.5% User Time 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout:21.3% System Time 2024-09-10T23:28:22.097 INFO:tasks.workunit.client.0.smithi067.stdout:21.8% CPU Utilization 2024-09-10T23:28:22.098 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .. 2024-09-10T23:28:22.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -r tmp ffsb 2024-09-10T23:28:22.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:21 smithi145 ceph-mon[34397]: pgmap v1317: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 359 KiB/s wr, 93 op/s 2024-09-10T23:28:22.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:21 smithi092 ceph-mon[34223]: pgmap v1317: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 359 KiB/s wr, 93 op/s 2024-09-10T23:28:24.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:23 smithi067 ceph-mon[30939]: pgmap v1318: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 385 KiB/s wr, 100 op/s 2024-09-10T23:28:24.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:23 smithi145 ceph-mon[34397]: pgmap v1318: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 385 KiB/s wr, 100 op/s 2024-09-10T23:28:24.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:23 smithi092 ceph-mon[34223]: pgmap v1318: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 385 KiB/s wr, 100 op/s 2024-09-10T23:28:25.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:24 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:28:25.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:24 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:28:25.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:24 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:28:26.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: pgmap v1319: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 379 KiB/s wr, 98 op/s 2024-09-10T23:28:26.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:28:26.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:28:26.137 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:25 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: pgmap v1319: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 379 KiB/s wr, 98 op/s 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:28:26.283 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:25 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: pgmap v1319: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 379 KiB/s wr, 98 op/s 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:28:26.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:25 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:28:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:27 smithi145 ceph-mon[34397]: pgmap v1320: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 192 KiB/s wr, 58 op/s 2024-09-10T23:28:28.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:27 smithi092 ceph-mon[34223]: pgmap v1320: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 192 KiB/s wr, 58 op/s 2024-09-10T23:28:28.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:27 smithi067 ceph-mon[30939]: pgmap v1320: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 192 KiB/s wr, 58 op/s 2024-09-10T23:28:30.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:29 smithi145 ceph-mon[34397]: pgmap v1321: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T23:28:30.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:29 smithi092 ceph-mon[34223]: pgmap v1321: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T23:28:30.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:29 smithi067 ceph-mon[30939]: pgmap v1321: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-09-10T23:28:31.742 INFO:tasks.quiescer.fs.[cephfs]:Total duration (56.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T23:28:31.743 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.45 2024-09-10T23:28:31.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:28:32.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:31 smithi145 ceph-mon[34397]: pgmap v1322: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 232 KiB/s wr, 56 op/s 2024-09-10T23:28:32.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:31 smithi092 ceph-mon[34223]: pgmap v1322: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 232 KiB/s wr, 56 op/s 2024-09-10T23:28:32.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:31 smithi067 ceph-mon[30939]: pgmap v1322: 129 pgs: 129 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 232 KiB/s wr, 56 op/s 2024-09-10T23:28:32.493 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:28:33.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:32 smithi067 ceph-mon[30939]: pgmap v1323: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 347 KiB/s wr, 79 op/s 2024-09-10T23:28:33.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:32 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:33.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:32 smithi092 ceph-mon[34223]: pgmap v1323: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 347 KiB/s wr, 79 op/s 2024-09-10T23:28:33.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:32 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:32 smithi145 ceph-mon[34397]: pgmap v1323: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 347 KiB/s wr, 79 op/s 2024-09-10T23:28:33.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:32 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.642+0000 7eff47e00640 1 -- 172.21.15.67:0/2255796943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff481091e0 msgr2=0x7eff481095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.642+0000 7eff47e00640 1 --2- 172.21.15.67:0/2255796943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff481091e0 0x7eff481095c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff38009900 tx=0x7eff3802a420 comp rx=0 tx=0).stop 2024-09-10T23:28:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 -- 172.21.15.67:0/2255796943 shutdown_connections 2024-09-10T23:28:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 --2- 172.21.15.67:0/2255796943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff4810f520 0x7eff48115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 --2- 172.21.15.67:0/2255796943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810eba0 0x7eff4810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 --2- 172.21.15.67:0/2255796943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff481091e0 0x7eff481095c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 -- 172.21.15.67:0/2255796943 >> 172.21.15.67:0/2255796943 conn(0x7eff48078830 msgr2=0x7eff4807ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 -- 172.21.15.67:0/2255796943 shutdown_connections 2024-09-10T23:28:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.644+0000 7eff47e00640 1 -- 172.21.15.67:0/2255796943 wait complete. 2024-09-10T23:28:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.648+0000 7eff47e00640 1 Processor -- start 2024-09-10T23:28:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.648+0000 7eff47e00640 1 -- start start 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 0x7eff4819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 0x7eff4819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 0x7eff481a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff48119170 con 0x7eff4810f520 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff48118ff0 con 0x7eff481091e0 2024-09-10T23:28:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.649+0000 7eff47e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff481192f0 con 0x7eff4810eba0 2024-09-10T23:28:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.650+0000 7eff46a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 0x7eff4819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.650+0000 7eff46a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 0x7eff4819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:39068/0 (socket says 172.21.15.67:39068) 2024-09-10T23:28:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.650+0000 7eff46a00640 1 -- 172.21.15.67:0/3554665351 learned_addr learned my addr 172.21.15.67:0/3554665351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.650+0000 7eff46000640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 0x7eff4819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.651+0000 7eff47400640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 0x7eff481a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.651+0000 7eff46000640 1 -- 172.21.15.67:0/3554665351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 msgr2=0x7eff4819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.651+0000 7eff46000640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 0x7eff4819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.652+0000 7eff46000640 1 -- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 msgr2=0x7eff481a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.652+0000 7eff46000640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 0x7eff481a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.652+0000 7eff46000640 1 -- 172.21.15.67:0/3554665351 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481a5750 con 0x7eff4810eba0 2024-09-10T23:28:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.652+0000 7eff47400640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 0x7eff481a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:28:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.652+0000 7eff46000640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 0x7eff4819fbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b3f0 tx=0x7eff3400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.654+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34015020 con 0x7eff4810eba0 2024-09-10T23:28:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.654+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff34011cc0 con 0x7eff4810eba0 2024-09-10T23:28:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.654+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34010cc0 con 0x7eff4810eba0 2024-09-10T23:28:33.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.656+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481a5a40 con 0x7eff4810eba0 2024-09-10T23:28:33.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.656+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481ac180 con 0x7eff4810eba0 2024-09-10T23:28:33.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.660+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff48109640 con 0x7eff4810eba0 2024-09-10T23:28:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.669+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7eff34011830 con 0x7eff4810eba0 2024-09-10T23:28:33.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.672+0000 7eff3f400640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 0x7eff2c0b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.673+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7eff340e5c00 con 0x7eff4810eba0 2024-09-10T23:28:33.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.675+0000 7eff46a00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 0x7eff2c0b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.677+0000 7eff46a00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 0x7eff2c0b7b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff380096c0 tx=0x7eff38000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:33.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:33.682+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7eff340ec080 con 0x7eff4810eba0 2024-09-10T23:28:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.156+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff481a07d0 con 0x7eff4810eba0 2024-09-10T23:28:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.157+0000 7eff3f400640 1 -- 172.21.15.67:0/3554665351 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11908 (secure 0 0 0) 0x7eff3401f020 con 0x7eff4810eba0 2024-09-10T23:28:34.159 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:28:34.159 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":44,"btime":"2024-09-10T23:28:32:840881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:28:32.795588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34730,34691,34631]},"id":1}]} 2024-09-10T23:28:34.160 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 44 2024-09-10T23:28:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.165+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 msgr2=0x7eff2c0b7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.165+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 0x7eff2c0b7b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff380096c0 tx=0x7eff38000bc0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.165+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 msgr2=0x7eff4819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.165+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 0x7eff4819fbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b3f0 tx=0x7eff3400b8b0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 shutdown_connections 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7eff2c0b56a0 0x7eff2c0b7b60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff4810f520 0x7eff481a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff4810eba0 0x7eff4819fbe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 --2- 172.21.15.67:0/3554665351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff481091e0 0x7eff4819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 >> 172.21.15.67:0/3554665351 conn(0x7eff48078830 msgr2=0x7eff4807ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 shutdown_connections 2024-09-10T23:28:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:34.166+0000 7eff47e00640 1 -- 172.21.15.67:0/3554665351 wait complete. 2024-09-10T23:28:34.523 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:34 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/3554665351' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:34.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:34 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/3554665351' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:34.631 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:34 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/3554665351' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:34.877 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T23:28:34.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T23:28:35.048 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-10T23:28:35.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T23:28:35.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:35 smithi067 ceph-mon[30939]: pgmap v1324: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 245 KiB/s wr, 56 op/s 2024-09-10T23:28:35.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:35 smithi092 ceph-mon[34223]: pgmap v1324: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 245 KiB/s wr, 56 op/s 2024-09-10T23:28:35.548 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db / --timeout 90 --expiration 148.45 --await-for 45.0' 2024-09-10T23:28:35.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db / --timeout 90 --expiration 148.45 --await-for 45.0 2024-09-10T23:28:35.629 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:35 smithi145 ceph-mon[34397]: pgmap v1324: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 245 KiB/s wr, 56 op/s 2024-09-10T23:28:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:28:38.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:37 smithi067 ceph-mon[30939]: pgmap v1325: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 385 KiB/s wr, 115 op/s 2024-09-10T23:28:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:37 smithi145 ceph-mon[34397]: pgmap v1325: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 385 KiB/s wr, 115 op/s 2024-09-10T23:28:38.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:37 smithi092 ceph-mon[34223]: pgmap v1325: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 385 KiB/s wr, 115 op/s 2024-09-10T23:28:38.939 DEBUG:teuthology.parallel:result is None 2024-09-10T23:28:38.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T23:28:39.085 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T23:28:39.086 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T23:28:39.098 INFO:tasks.quiescer:joining Quiescers 2024-09-10T23:28:39.099 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T23:28:39.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.747+0000 7ff24e400640 1 -- 172.21.15.67:0/1743063566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff24810aa70 msgr2=0x7ff2481154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.747+0000 7ff24e400640 1 --2- 172.21.15.67:0/1743063566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff24810aa70 0x7ff2481154d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff234009900 tx=0x7ff23402a490 comp rx=0 tx=0).stop 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.748+0000 7ff24e400640 1 -- 172.21.15.67:0/1743063566 shutdown_connections 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.748+0000 7ff24e400640 1 --2- 172.21.15.67:0/1743063566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff248115a10 0x7ff248117e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.748+0000 7ff24e400640 1 --2- 172.21.15.67:0/1743063566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff24810aa70 0x7ff2481154d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.748+0000 7ff24e400640 1 --2- 172.21.15.67:0/1743063566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff24810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.748+0000 7ff24e400640 1 -- 172.21.15.67:0/1743063566 >> 172.21.15.67:0/1743063566 conn(0x7ff24806dfc0 msgr2=0x7ff24806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.749+0000 7ff24e400640 1 -- 172.21.15.67:0/1743063566 shutdown_connections 2024-09-10T23:28:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.749+0000 7ff24e400640 1 -- 172.21.15.67:0/1743063566 wait complete. 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.750+0000 7ff24e400640 1 Processor -- start 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.750+0000 7ff24e400640 1 -- start start 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff24e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff248114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff24e400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff24e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff24e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff24811b4b0 con 0x7ff24810a150 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff24e400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff24811b330 con 0x7ff24810aa70 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff246a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff247e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff246a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:45124/0 (socket says 172.21.15.67:45124) 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff247e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40880/0 (socket says 172.21.15.67:40880) 2024-09-10T23:28:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.752+0000 7ff246a00640 1 -- 172.21.15.67:0/176512472 learned_addr learned my addr 172.21.15.67:0/176512472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.753+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff24811b630 con 0x7ff248115a10 2024-09-10T23:28:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.753+0000 7ff247400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff248114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247400640 1 -- 172.21.15.67:0/176512472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 msgr2=0x7ff2481ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247400640 1 -- 172.21.15.67:0/176512472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 msgr2=0x7ff248114e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247400640 1 -- 172.21.15.67:0/176512472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2481ae150 con 0x7ff24810a150 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff247e00640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.754+0000 7ff246a00640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:28:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.755+0000 7ff247400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff248114920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b3f0 tx=0x7ff23800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.756+0000 7ff23fe00640 1 -- 172.21.15.67:0/176512472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238015020 con 0x7ff24810a150 2024-09-10T23:28:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.756+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2481ae440 con 0x7ff24810a150 2024-09-10T23:28:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.756+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481b4cd0 con 0x7ff24810a150 2024-09-10T23:28:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.757+0000 7ff23fe00640 1 -- 172.21.15.67:0/176512472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff238011cc0 con 0x7ff24810a150 2024-09-10T23:28:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.757+0000 7ff23fe00640 1 -- 172.21.15.67:0/176512472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238010cc0 con 0x7ff24810a150 2024-09-10T23:28:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.765+0000 7ff23fe00640 1 -- 172.21.15.67:0/176512472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff238011830 con 0x7ff24810a150 2024-09-10T23:28:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.768+0000 7ff23fe00640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 0x7ff22c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.768+0000 7ff23fe00640 1 -- 172.21.15.67:0/176512472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7ff2380e5ad0 con 0x7ff24810a150 2024-09-10T23:28:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.769+0000 7ff246a00640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 0x7ff22c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.771+0000 7ff246a00640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 0x7ff22c0b7990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2340096c0 tx=0x7ff234005e60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.784+0000 7ff250fce740 1 Processor -- start 2024-09-10T23:28:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 -- start start 2024-09-10T23:28:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5bc40 0x55cec4c5a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cec4bcfa90 con 0x55cec4c5a6e0 2024-09-10T23:28:39.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cec4bbeec0 con 0x55cec4c66510 2024-09-10T23:28:39.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.785+0000 7ff250fce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cec4bc2cd0 con 0x55cec4c5bc40 2024-09-10T23:28:39.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:45134/0 (socket says 172.21.15.67:45134) 2024-09-10T23:28:39.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 -- 172.21.15.67:0/3696897775 learned_addr learned my addr 172.21.15.67:0/3696897775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff246a00640 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 -- 172.21.15.67:0/3696897775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5bc40 msgr2=0x55cec4c5a1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5bc40 0x55cec4c5a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 -- 172.21.15.67:0/3696897775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 msgr2=0x55cec4c5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.786+0000 7ff247e00640 1 -- 172.21.15.67:0/3696897775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec4b649d0 con 0x55cec4c66510 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.787+0000 7ff247e00640 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff2400117e0 tx=0x7ff240011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4a207fbc116145 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.788+0000 7ff23cc00640 1 -- 172.21.15.67:0/3696897775 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240011310 con 0x55cec4c66510 2024-09-10T23:28:39.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.788+0000 7ff23cc00640 1 -- 172.21.15.67:0/3696897775 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff240003e00 con 0x55cec4c66510 2024-09-10T23:28:39.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.788+0000 7ff23cc00640 1 -- 172.21.15.67:0/3696897775 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2400044f0 con 0x55cec4c66510 2024-09-10T23:28:39.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 -- 172.21.15.67:0/3696897775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 msgr2=0x55cec4c68900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff2400117e0 tx=0x7ff240011ca0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 -- 172.21.15.67:0/3696897775 shutdown_connections 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c68900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 --2- 172.21.15.67:0/3696897775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5bc40 0x55cec4c5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 -- 172.21.15.67:0/3696897775 >> 172.21.15.67:0/3696897775 conn(0x55cec4c246f0 msgr2=0x55cec4c57fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.789+0000 7ff250fce740 1 -- 172.21.15.67:0/3696897775 shutdown_connections 2024-09-10T23:28:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.790+0000 7ff250fce740 1 -- 172.21.15.67:0/3696897775 wait complete. 2024-09-10T23:28:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.791+0000 7ff250fce740 1 -- start start 2024-09-10T23:28:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.791+0000 7ff250fce740 1 Processor -- start 2024-09-10T23:28:39.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.792+0000 7ff250fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c63560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55cec4a207b0 con 0x55cec4c5a6e0 2024-09-10T23:28:39.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cec4b63650 con 0x55cec4c5bc40 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cec4c69330 con 0x55cec4c66510 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.793+0000 7ff250fce740 10 client.0 _ll_register_callbacks cb 0x55cec48ca0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff246a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff247e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff246a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:45136/0 (socket says 172.21.15.67:45136) 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff247e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40894/0 (socket says 172.21.15.67:40894) 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff246a00640 1 -- 172.21.15.67:0/2435591105 learned_addr learned my addr 172.21.15.67:0/2435591105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.794+0000 7ff247400640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c63560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 -- 172.21.15.67:0/2435591105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 msgr2=0x55cec4c63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 -- 172.21.15.67:0/2435591105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 msgr2=0x55cec4c63560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c63560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 -- 172.21.15.67:0/2435591105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec492cba0 con 0x55cec4c5bc40 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff247e00640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff247400640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.795+0000 7ff246a00640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2340057c0 tx=0x7ff234009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f943f405d544dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.796+0000 7ff231600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.796+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23404d070 con 0x55cec4c5bc40 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.796+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7ff23403ca50 con 0x55cec4c5bc40 2024-09-10T23:28:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.796+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff240003e00 con 0x55cec4c5bc40 2024-09-10T23:28:39.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.806+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff234057050 con 0x55cec4c5bc40 2024-09-10T23:28:39.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.806+0000 7ff250fce740 10 client.36341 fetch_fsmap learned FSMap version 45 2024-09-10T23:28:39.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.807+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cec4c75030 con 0x55cec4c5bc40 2024-09-10T23:28:39.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.807+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff240003e00 con 0x55cec4c5bc40 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7ff234052440 con 0x55cec4c5bc40 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 10 client.36341 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 10 client.36341 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 20 client.36341 populate_metadata read hostname 'smithi067' 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4c76590 0x55cec4c78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 4 client.36341 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:28:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec4bd19e0 con 0x55cec4c76590 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.808+0000 7ff247400640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4c76590 0x55cec4c78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.810+0000 7ff247400640 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4c76590 0x55cec4c78a00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f9af9a2d0af0c4 server_cookie=6d034e4d3407c152 in_seq=0 out_seq=0 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.810+0000 7ff231600640 10 client.36341 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.811+0000 7ff231600640 1 -- 172.21.15.67:0/2435591105 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cec4bd19e0 con 0x55cec4c76590 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.811+0000 7ff231600640 10 client.36341 handle_command_reply: tid=0 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.812+0000 7ff250fce740 1 client.36341 shutdown 2024-09-10T23:28:39.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 msgr2=0x55cec4c63aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2340057c0 tx=0x7ff234009260 comp rx=0 tx=0).stop 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 shutdown_connections 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4c76590 0x55cec4c78a00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c66510 0x55cec4c63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c5bc40 0x55cec4c63aa0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2340057c0 tx=0x7ff234009260 comp rx=0 tx=0).stop 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2435591105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c5a6e0 0x55cec4c63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 >> 172.21.15.67:0/2435591105 conn(0x55cec4c246f0 msgr2=0x55cec4c5db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.813+0000 7ff231600640 0 client.36341 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:28:39.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.816+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 shutdown_connections 2024-09-10T23:28:39.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.817+0000 7ff250fce740 1 -- 172.21.15.67:0/2435591105 wait complete. 2024-09-10T23:28:39.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.817+0000 7ff250fce740 20 client.36341 trim_cache size 0 max 16384 2024-09-10T23:28:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.857+0000 7ff250fce740 1 Processor -- start 2024-09-10T23:28:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.857+0000 7ff250fce740 1 -- start start 2024-09-10T23:28:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c60790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff228012980 con 0x55cec4c57de0 2024-09-10T23:28:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2280101c0 con 0x55cec4c66510 2024-09-10T23:28:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff250fce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff22800e030 con 0x55cec4c5a6e0 2024-09-10T23:28:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:28:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff247400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff247400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40904/0 (socket says 172.21.15.67:40904) 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.858+0000 7ff247400640 1 -- 172.21.15.67:0/2352031625 learned_addr learned my addr 172.21.15.67:0/2352031625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.859+0000 7ff246a00640 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff247400640 1 -- 172.21.15.67:0/2352031625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 msgr2=0x55cec4c578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff247400640 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff247400640 1 -- 172.21.15.67:0/2352031625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 msgr2=0x55cec4c60790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff247400640 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c60790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff247400640 1 -- 172.21.15.67:0/2352031625 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec4c53900 con 0x55cec4c5a6e0 2024-09-10T23:28:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.860+0000 7ff246a00640 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:28:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.861+0000 7ff247400640 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2380ab870 tx=0x7ff23800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0cb3f207c34383 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.861+0000 7ff233e00640 1 -- 172.21.15.67:0/2352031625 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380d2020 con 0x55cec4c5a6e0 2024-09-10T23:28:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.861+0000 7ff233e00640 1 -- 172.21.15.67:0/2352031625 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380ab680 con 0x55cec4c5a6e0 2024-09-10T23:28:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.861+0000 7ff233e00640 1 -- 172.21.15.67:0/2352031625 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380a7de0 con 0x55cec4c5a6e0 2024-09-10T23:28:39.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.862+0000 7ff250fce740 1 -- 172.21.15.67:0/2352031625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 msgr2=0x55cec4c59de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:39.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.862+0000 7ff250fce740 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2380ab870 tx=0x7ff23800bf60 comp rx=0 tx=0).stop 2024-09-10T23:28:39.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.863+0000 7ff250fce740 1 -- 172.21.15.67:0/2352031625 shutdown_connections 2024-09-10T23:28:39.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.864+0000 7ff250fce740 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c60790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.864+0000 7ff250fce740 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.864+0000 7ff250fce740 1 --2- 172.21.15.67:0/2352031625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c59de0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.864+0000 7ff250fce740 1 -- 172.21.15.67:0/2352031625 >> 172.21.15.67:0/2352031625 conn(0x55cec4c246f0 msgr2=0x55cec4c61de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.864+0000 7ff250fce740 1 -- 172.21.15.67:0/2352031625 shutdown_connections 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.865+0000 7ff250fce740 1 -- 172.21.15.67:0/2352031625 wait complete. 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.865+0000 7ff250fce740 1 -- start start 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.865+0000 7ff250fce740 1 Processor -- start 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.867+0000 7ff250fce740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff22800e680 con 0x55cec4c57de0 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff22800faf0 con 0x55cec4c66510 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff22800ed50 con 0x55cec4c5a6e0 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff250fce740 10 client.0 _ll_register_callbacks cb 0x7ff234040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.868+0000 7ff246a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff246a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:40910/0 (socket says 172.21.15.67:40910) 2024-09-10T23:28:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff246a00640 1 -- 172.21.15.67:0/2716522466 learned_addr learned my addr 172.21.15.67:0/2716522466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff247e00640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff247400640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff246a00640 1 -- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 msgr2=0x55cec4c6ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff246a00640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c6ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.869+0000 7ff246a00640 1 -- 172.21.15.67:0/2716522466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 msgr2=0x55cec4c61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:28:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.870+0000 7ff246a00640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c61020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.870+0000 7ff246a00640 1 -- 172.21.15.67:0/2716522466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec4c5f780 con 0x55cec4c5a6e0 2024-09-10T23:28:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.870+0000 7ff247400640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:28:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.871+0000 7ff246a00640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2340528b0 tx=0x7ff234052cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486e7d6d7a209b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.871+0000 7ff231600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.872+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23404d040 con 0x55cec4c5a6e0 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.872+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7ff23403cc60 con 0x55cec4c5a6e0 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.872+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2380ab680 con 0x55cec4c5a6e0 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.873+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff23406d050 con 0x55cec4c5a6e0 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.873+0000 7ff250fce740 10 client.26311 fetch_fsmap learned FSMap version 45 2024-09-10T23:28:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.873+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cec4ce4670 con 0x55cec4c5a6e0 2024-09-10T23:28:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380ab680 con 0x55cec4c5a6e0 2024-09-10T23:28:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7ff234049c90 con 0x55cec4c5a6e0 2024-09-10T23:28:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 10 client.26311 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:28:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 10 client.26311 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:28:39.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 20 client.26311 populate_metadata read hostname 'smithi067' 2024-09-10T23:28:39.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4ce4ea0 0x55cec4ce72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:39.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 4 client.26311 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.45, "await_for": 45.0}] 2024-09-10T23:28:39.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.875+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.45, "await_for": 45.0}) -- 0x55cec4bcd490 con 0x55cec4ce4ea0 2024-09-10T23:28:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.876+0000 7ff247400640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4ce4ea0 0x55cec4ce72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:39.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.877+0000 7ff247400640 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4ce4ea0 0x55cec4ce72c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6929241673613d4 server_cookie=bd9e3409445eb6f0 in_seq=0 out_seq=0 2024-09-10T23:28:39.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:39.878+0000 7ff231600640 10 client.26311 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:28:40.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:39 smithi067 ceph-mon[30939]: pgmap v1326: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-10T23:28:40.035 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:39 smithi067 ceph-mon[30939]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:39 smithi145 ceph-mon[34397]: pgmap v1326: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-10T23:28:40.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:39 smithi145 ceph-mon[34397]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:40.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:39 smithi092 ceph-mon[34223]: pgmap v1326: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-10T23:28:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:39 smithi092 ceph-mon[34223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 -- 172.21.15.67:0/956098087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4340104fa0 msgr2=0x7f4340105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 --2- 172.21.15.67:0/956098087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4340104fa0 0x7f4340105380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4330009900 tx=0x7f433002a410 comp rx=0 tx=0).stop 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 -- 172.21.15.67:0/956098087 shutdown_connections 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 --2- 172.21.15.67:0/956098087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434010b330 0x7f4340111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 --2- 172.21.15.67:0/956098087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010a9b0 0x7f434010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 --2- 172.21.15.67:0/956098087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4340104fa0 0x7f4340105380 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 -- 172.21.15.67:0/956098087 >> 172.21.15.67:0/956098087 conn(0x7f4340100a50 msgr2=0x7f4340102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.664+0000 7f4345a00640 1 -- 172.21.15.67:0/956098087 shutdown_connections 2024-09-10T23:28:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.665+0000 7f4345a00640 1 -- 172.21.15.67:0/956098087 wait complete. 2024-09-10T23:28:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.666+0000 7f4345a00640 1 Processor -- start 2024-09-10T23:28:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.666+0000 7f4345a00640 1 -- start start 2024-09-10T23:28:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4340104fa0 0x7f434019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 0x7f434019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 0x7f43401a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4340114e10 con 0x7f434010a9b0 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4340114c90 con 0x7f434010b330 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.667+0000 7f4345a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4340114f90 con 0x7f4340104fa0 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 0x7f434019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f433f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 0x7f43401a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f433f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 0x7f43401a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:45170/0 (socket says 172.21.15.67:45170) 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f433f400640 1 -- 172.21.15.67:0/2249077881 learned_addr learned my addr 172.21.15.67:0/2249077881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 -- 172.21.15.67:0/2249077881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4340104fa0 msgr2=0x7f434019b420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4340104fa0 0x7f434019b420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 -- 172.21.15.67:0/2249077881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 msgr2=0x7f43401a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 0x7f43401a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.668+0000 7f4337e00640 1 -- 172.21.15.67:0/2249077881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43401a14d0 con 0x7f434010a9b0 2024-09-10T23:28:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.669+0000 7f4337e00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 0x7f434019b960 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.673+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432800ccc0 con 0x7f434010a9b0 2024-09-10T23:28:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.673+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43280108c0 con 0x7f434010a9b0 2024-09-10T23:28:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.673+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328018810 con 0x7f434010a9b0 2024-09-10T23:28:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.675+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a17c0 con 0x7f434010a9b0 2024-09-10T23:28:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.675+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434006cb70 con 0x7f434010a9b0 2024-09-10T23:28:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.676+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4340106250 con 0x7f434010a9b0 2024-09-10T23:28:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.687+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4328010430 con 0x7f434010a9b0 2024-09-10T23:28:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.690+0000 7f4337400640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 0x7f43240b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:28:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.690+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7f43280e6d50 con 0x7f434010a9b0 2024-09-10T23:28:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.691+0000 7f433ea00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 0x7f43240b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:28:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.696+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f43280ad570 con 0x7f434010a9b0 2024-09-10T23:28:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:40.699+0000 7f433ea00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 0x7f43240b7a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f433002a930 tx=0x7f4330000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:28:41.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.122+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4340105380 con 0x7f434010a9b0 2024-09-10T23:28:41.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.124+0000 7f4337400640 1 -- 172.21.15.67:0/2249077881 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11910 (secure 0 0 0) 0x7f43280ace70 con 0x7f434010a9b0 2024-09-10T23:28:41.126 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:28:41.126 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"btime":"2024-09-10T23:28:38:783141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:28:38.628606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:28:41.126 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-09-10T23:28:41.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.128+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 msgr2=0x7f43240b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:41.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.128+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 0x7f43240b7a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f433002a930 tx=0x7f4330000bc0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.128+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 msgr2=0x7f434019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:41.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.128+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 0x7f434019b960 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 shutdown_connections 2024-09-10T23:28:41.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f43240b55a0 0x7f43240b7a60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434010b330 0x7f43401a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f434010a9b0 0x7f434019b960 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 --2- 172.21.15.67:0/2249077881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4340104fa0 0x7f434019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.130+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 >> 172.21.15.67:0/2249077881 conn(0x7f4340100a50 msgr2=0x7f4340101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.131+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 shutdown_connections 2024-09-10T23:28:41.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:41.131+0000 7f4345a00640 1 -- 172.21.15.67:0/2249077881 wait complete. 2024-09-10T23:28:42.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:41 smithi145 ceph-mon[34397]: pgmap v1327: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 389 KiB/s wr, 136 op/s 2024-09-10T23:28:42.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:41 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2249077881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:41 smithi092 ceph-mon[34223]: pgmap v1327: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 389 KiB/s wr, 136 op/s 2024-09-10T23:28:42.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:41 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2249077881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:41 smithi067 ceph-mon[30939]: pgmap v1327: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 389 KiB/s wr, 136 op/s 2024-09-10T23:28:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:41 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2249077881' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:28:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.812+0000 7ff231600640 1 -- 172.21.15.67:0/2716522466 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55cec4bcd490 con 0x55cec4ce4ea0 2024-09-10T23:28:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.812+0000 7ff231600640 10 client.26311 handle_command_reply: tid=0 2024-09-10T23:28:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.812+0000 7ff250fce740 1 client.26311 shutdown 2024-09-10T23:28:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 msgr2=0x55cec4c6e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2340528b0 tx=0x7ff234052cd0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 shutdown_connections 2024-09-10T23:28:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x55cec4ce4ea0 0x55cec4ce72c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cec4c66510 0x55cec4c6ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cec4c5a6e0 0x55cec4c6e650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff231600640 0 client.26311 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.813+0000 7ff250fce740 1 --2- 172.21.15.67:0/2716522466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cec4c57de0 0x55cec4c61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.814+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 >> 172.21.15.67:0/2716522466 conn(0x55cec4c246f0 msgr2=0x55cec4c64b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.814+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 shutdown_connections 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.814+0000 7ff250fce740 1 -- 172.21.15.67:0/2716522466 wait complete. 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.814+0000 7ff250fce740 20 client.26311 trim_cache size 0 max 16384 2024-09-10T23:28:42.817 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 45, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 86, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "eff1017": { 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 86, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 0.0, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:28:42.818 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 148.4, 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": false, 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:28:42.819 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.817+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 msgr2=0x7ff22c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.817+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 0x7ff22c0b7990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2340096c0 tx=0x7ff234005e60 comp rx=0 tx=0).stop 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.817+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 msgr2=0x7ff248114920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.817+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff248114920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b3f0 tx=0x7ff23800b8b0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 shutdown_connections 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff22c0b54d0 0x7ff22c0b7990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff248115a10 0x7ff2481ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24810aa70 0x7ff248114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 --2- 172.21.15.67:0/176512472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24810a150 0x7ff248114920 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:28:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.818+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 >> 172.21.15.67:0/176512472 conn(0x7ff24806dfc0 msgr2=0x7ff24810b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:28:42.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.826+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 shutdown_connections 2024-09-10T23:28:42.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:28:42.826+0000 7ff24e400640 1 -- 172.21.15.67:0/176512472 wait complete. 2024-09-10T23:28:44.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:43 smithi145 ceph-mon[34397]: pgmap v1328: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 309 KiB/s wr, 161 op/s 2024-09-10T23:28:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:43 smithi092 ceph-mon[34223]: pgmap v1328: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 309 KiB/s wr, 161 op/s 2024-09-10T23:28:44.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:43 smithi067 ceph-mon[30939]: pgmap v1328: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 309 KiB/s wr, 161 op/s 2024-09-10T23:28:45.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:45 smithi145 ceph-mon[34397]: pgmap v1329: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 152 KiB/s wr, 130 op/s 2024-09-10T23:28:45.520 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:45 smithi092 ceph-mon[34223]: pgmap v1329: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 152 KiB/s wr, 130 op/s 2024-09-10T23:28:45.548 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:45 smithi067 ceph-mon[30939]: pgmap v1329: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 152 KiB/s wr, 130 op/s 2024-09-10T23:28:47.045 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --cancel --all' 2024-09-10T23:28:47.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --cancel --all 2024-09-10T23:28:48.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:47 smithi067 ceph-mon[30939]: pgmap v1330: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 163 op/s 2024-09-10T23:28:48.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:47 smithi145 ceph-mon[34397]: pgmap v1330: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 163 op/s 2024-09-10T23:28:48.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:47 smithi092 ceph-mon[34223]: pgmap v1330: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 167 KiB/s wr, 163 op/s 2024-09-10T23:28:50.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:49 smithi067 ceph-mon[30939]: pgmap v1331: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 123 op/s 2024-09-10T23:28:50.135 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:49 smithi145 ceph-mon[34397]: pgmap v1331: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 123 op/s 2024-09-10T23:28:50.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:49 smithi092 ceph-mon[34223]: pgmap v1331: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 123 op/s 2024-09-10T23:28:51.948 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:51 smithi067 ceph-mon[30939]: pgmap v1332: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 124 op/s 2024-09-10T23:28:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:51 smithi145 ceph-mon[34397]: pgmap v1332: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 124 op/s 2024-09-10T23:28:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:51 smithi092 ceph-mon[34223]: pgmap v1332: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 124 op/s 2024-09-10T23:28:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:53 smithi145 ceph-mon[34397]: pgmap v1333: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 136 op/s 2024-09-10T23:28:54.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:53 smithi092 ceph-mon[34223]: pgmap v1333: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 136 op/s 2024-09-10T23:28:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:53 smithi067 ceph-mon[30939]: pgmap v1333: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 136 op/s 2024-09-10T23:28:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:55 smithi145 ceph-mon[34397]: pgmap v1334: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 114 op/s 2024-09-10T23:28:56.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:55 smithi092 ceph-mon[34223]: pgmap v1334: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 114 op/s 2024-09-10T23:28:56.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:55 smithi067 ceph-mon[30939]: pgmap v1334: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 114 op/s 2024-09-10T23:28:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:57 smithi145 ceph-mon[34397]: pgmap v1335: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 138 op/s 2024-09-10T23:28:58.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:57 smithi092 ceph-mon[34223]: pgmap v1335: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 138 op/s 2024-09-10T23:28:58.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:57 smithi067 ceph-mon[30939]: pgmap v1335: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 138 op/s 2024-09-10T23:29:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:28:59 smithi145 ceph-mon[34397]: pgmap v1336: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 106 op/s 2024-09-10T23:29:00.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:28:59 smithi092 ceph-mon[34223]: pgmap v1336: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 106 op/s 2024-09-10T23:29:00.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:28:59 smithi067 ceph-mon[30939]: pgmap v1336: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 106 op/s 2024-09-10T23:29:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:01 smithi145 ceph-mon[34397]: pgmap v1337: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 88 op/s 2024-09-10T23:29:02.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:01 smithi092 ceph-mon[34223]: pgmap v1337: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 88 op/s 2024-09-10T23:29:02.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:01 smithi067 ceph-mon[30939]: pgmap v1337: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 88 op/s 2024-09-10T23:29:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:03 smithi145 ceph-mon[34397]: pgmap v1338: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 29 KiB/s wr, 75 op/s 2024-09-10T23:29:04.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:03 smithi092 ceph-mon[34223]: pgmap v1338: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 29 KiB/s wr, 75 op/s 2024-09-10T23:29:04.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:03 smithi067 ceph-mon[30939]: pgmap v1338: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 29 KiB/s wr, 75 op/s 2024-09-10T23:29:05.938 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eff1017', quiesce took 34.2 seconds. Will release after: -5.7500000000000036 2024-09-10T23:29:05.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:29:05.991 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:06.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:05 smithi145 ceph-mon[34397]: pgmap v1339: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 64 op/s 2024-09-10T23:29:06.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:05 smithi067 ceph-mon[30939]: pgmap v1339: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 64 op/s 2024-09-10T23:29:06.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:05 smithi092 ceph-mon[34223]: pgmap v1339: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 64 op/s 2024-09-10T23:29:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 -- 172.21.15.67:0/1121668519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0104fb0 msgr2=0x7efdb0105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 --2- 172.21.15.67:0/1121668519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0104fb0 0x7efdb0105390 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efda0009ca0 tx=0x7efda002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 -- 172.21.15.67:0/1121668519 shutdown_connections 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 --2- 172.21.15.67:0/1121668519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010b340 0x7efdb0111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 --2- 172.21.15.67:0/1121668519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb010a9c0 0x7efdb010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.334+0000 7efdb6600640 1 --2- 172.21.15.67:0/1121668519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb0104fb0 0x7efdb0105390 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.335+0000 7efdb6600640 1 -- 172.21.15.67:0/1121668519 >> 172.21.15.67:0/1121668519 conn(0x7efdb0100a00 msgr2=0x7efdb0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.335+0000 7efdb6600640 1 -- 172.21.15.67:0/1121668519 shutdown_connections 2024-09-10T23:29:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.335+0000 7efdb6600640 1 -- 172.21.15.67:0/1121668519 wait complete. 2024-09-10T23:29:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 Processor -- start 2024-09-10T23:29:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 -- start start 2024-09-10T23:29:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb0104fb0 0x7efdb019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 0x7efdb019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0114e50 con 0x7efdb010b340 2024-09-10T23:29:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0114cd0 con 0x7efdb0104fb0 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.336+0000 7efdb6600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0114fd0 con 0x7efdb010a9c0 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38574/0 (socket says 172.21.15.67:38574) 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 -- 172.21.15.67:0/4164862926 learned_addr learned my addr 172.21.15.67:0/4164862926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdaf400640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb0104fb0 0x7efdb019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efda6a00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 0x7efdb019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 -- 172.21.15.67:0/4164862926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 msgr2=0x7efdb019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 0x7efdb019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 -- 172.21.15.67:0/4164862926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb0104fb0 msgr2=0x7efdb019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb0104fb0 0x7efdb019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 -- 172.21.15.67:0/4164862926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb01a14b0 con 0x7efdb010b340 2024-09-10T23:29:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efda6a00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 0x7efdb019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.337+0000 7efdafe00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efda800d950 tx=0x7efda800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.338+0000 7efdacc00640 1 -- 172.21.15.67:0/4164862926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8017030 con 0x7efdb010b340 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.338+0000 7efdacc00640 1 -- 172.21.15.67:0/4164862926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda8004020 con 0x7efdb010b340 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.338+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb01a17a0 con 0x7efdb010b340 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.338+0000 7efdacc00640 1 -- 172.21.15.67:0/4164862926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8003ab0 con 0x7efdb010b340 2024-09-10T23:29:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.338+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb01a7e60 con 0x7efdb010b340 2024-09-10T23:29:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.342+0000 7efdacc00640 1 -- 172.21.15.67:0/4164862926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efda8003c50 con 0x7efdb010b340 2024-09-10T23:29:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.344+0000 7efdacc00640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 0x7efd940b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.344+0000 7efdacc00640 1 -- 172.21.15.67:0/4164862926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7efda80e5f50 con 0x7efdb010b340 2024-09-10T23:29:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.344+0000 7efdaf400640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 0x7efd940b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.345+0000 7efdaf400640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 0x7efd940b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efda0006010 tx=0x7efda0031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.364+0000 7efdb9258740 1 Processor -- start 2024-09-10T23:29:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 -- start start 2024-09-10T23:29:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd0a0 0x5636c07bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 0x5636c07c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5636c0734ff0 con 0x5636c07c9950 2024-09-10T23:29:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636c0724420 con 0x5636c07bd0a0 2024-09-10T23:29:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.365+0000 7efdb9258740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5636c0728230 con 0x5636c07bd9c0 2024-09-10T23:29:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efdafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efdafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38586/0 (socket says 172.21.15.67:38586) 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 0x5636c07c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efdafe00640 1 -- 172.21.15.67:0/3475848984 learned_addr learned my addr 172.21.15.67:0/3475848984 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efdaf400640 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd0a0 0x5636c07bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 -- 172.21.15.67:0/3475848984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd0a0 msgr2=0x5636c07bd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd0a0 0x5636c07bd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 -- 172.21.15.67:0/3475848984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 msgr2=0x5636c07cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efda6a00640 1 -- 172.21.15.67:0/3475848984 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c04828b0 con 0x5636c07bd9c0 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.366+0000 7efdafe00640 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:29:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.367+0000 7efda6a00640 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 0x5636c07c9410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd9800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f773b179bb9dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.367+0000 7efda4c00640 1 -- 172.21.15.67:0/3475848984 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98016070 con 0x5636c07bd9c0 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.367+0000 7efda4c00640 1 -- 172.21.15.67:0/3475848984 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd980054b0 con 0x5636c07bd9c0 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.367+0000 7efda4c00640 1 -- 172.21.15.67:0/3475848984 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98004da0 con 0x5636c07bd9c0 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 -- 172.21.15.67:0/3475848984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 msgr2=0x5636c07c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 0x5636c07c9410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd9800e6c0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 -- 172.21.15.67:0/3475848984 shutdown_connections 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07c9950 0x5636c07cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07bd9c0 0x5636c07c9410 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 --2- 172.21.15.67:0/3475848984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd0a0 0x5636c07bd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 -- 172.21.15.67:0/3475848984 >> 172.21.15.67:0/3475848984 conn(0x5636c07baed0 msgr2=0x5636c07bc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.368+0000 7efdb9258740 1 -- 172.21.15.67:0/3475848984 shutdown_connections 2024-09-10T23:29:07.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.369+0000 7efdb9258740 1 -- 172.21.15.67:0/3475848984 wait complete. 2024-09-10T23:29:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.369+0000 7efdb9258740 1 -- start start 2024-09-10T23:29:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 Processor -- start 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5636c0585810 con 0x5636c07bd0a0 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636c06bf550 con 0x5636c07bd9c0 2024-09-10T23:29:07.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5636c07cc600 con 0x5636c07c9950 2024-09-10T23:29:07.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.370+0000 7efdb9258740 10 client.0 _ll_register_callbacks cb 0x5636c06cf040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:29:07.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38594/0 (socket says 172.21.15.67:38594) 2024-09-10T23:29:07.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efdaf400640 1 -- 172.21.15.67:0/1300166655 learned_addr learned my addr 172.21.15.67:0/1300166655 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:07.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efdafe00640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 -- 172.21.15.67:0/1300166655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 msgr2=0x5636c07c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 -- 172.21.15.67:0/1300166655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 msgr2=0x5636c07c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.371+0000 7efda6a00640 1 -- 172.21.15.67:0/1300166655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c06b1200 con 0x5636c07bd9c0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.372+0000 7efda6a00640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07c6720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd9800efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d34583a1a4b986 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.372+0000 7efd9d600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.372+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98016040 con 0x5636c07bd9c0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.372+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd980054b0 con 0x5636c07bd9c0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.372+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7efd98004020 con 0x5636c07bd9c0 2024-09-10T23:29:07.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efd9802e050 con 0x5636c07bd9c0 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efdb9258740 10 client.36365 fetch_fsmap learned FSMap version 45 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5636c07d7fd0 con 0x5636c07bd9c0 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd980054b0 con 0x5636c07bd9c0 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7efd9800f040 con 0x5636c07bd9c0 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efdb9258740 10 client.36365 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.373+0000 7efdb9258740 10 client.36365 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.374+0000 7efdb9258740 20 client.36365 populate_metadata read hostname 'smithi067' 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.374+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c07d9380 0x5636c07db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.374+0000 7efdb9258740 4 client.36365 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:29:07.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.374+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c0736f40 con 0x5636c07d9380 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.374+0000 7efdaf400640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c07d9380 0x5636c07db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.375+0000 7efdaf400640 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c07d9380 0x5636c07db770 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fef83b12ba20359 server_cookie=5b0070b687cf5c32 in_seq=0 out_seq=0 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.375+0000 7efd9d600640 10 client.36365 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.375+0000 7efd9d600640 1 -- 172.21.15.67:0/1300166655 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636c0736f40 con 0x5636c07d9380 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.375+0000 7efd9d600640 10 client.36365 handle_command_reply: tid=0 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 client.36365 shutdown 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 msgr2=0x5636c07c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07c6720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd9800efd0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 shutdown_connections 2024-09-10T23:29:07.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c07d9380 0x5636c07db770 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07c6720 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 --2- 172.21.15.67:0/1300166655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 >> 172.21.15.67:0/1300166655 conn(0x5636c07baed0 msgr2=0x5636c07c0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efd9d600640 0 client.36365 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 shutdown_connections 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 1 -- 172.21.15.67:0/1300166655 wait complete. 2024-09-10T23:29:07.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.376+0000 7efdb9258740 20 client.36365 trim_cache size 0 max 16384 2024-09-10T23:29:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 Processor -- start 2024-09-10T23:29:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 -- start start 2024-09-10T23:29:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b4af0 con 0x5636c07bd0a0 2024-09-10T23:29:07.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b2330 con 0x5636c07bd9c0 2024-09-10T23:29:07.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdb9258740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b01a0 con 0x5636c07c9950 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38598/0 (socket says 172.21.15.67:38598) 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.405+0000 7efdaf400640 1 -- 172.21.15.67:0/4228988781 learned_addr learned my addr 172.21.15.67:0/4228988781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdafe00640 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efda6a00640 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 -- 172.21.15.67:0/4228988781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 msgr2=0x5636c07cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 -- 172.21.15.67:0/4228988781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 msgr2=0x5636c07cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 -- 172.21.15.67:0/4228988781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c07bd7f0 con 0x5636c07bd0a0 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efdaf400640 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efda0009970 tx=0x7efda003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9072ffce9a26fce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.406+0000 7efd9fe00640 1 -- 172.21.15.67:0/4228988781 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0030a90 con 0x5636c07bd0a0 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.407+0000 7efd9fe00640 1 -- 172.21.15.67:0/4228988781 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda0030c30 con 0x5636c07bd0a0 2024-09-10T23:29:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.407+0000 7efd9fe00640 1 -- 172.21.15.67:0/4228988781 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0053cc0 con 0x5636c07bd0a0 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.407+0000 7efdb9258740 1 -- 172.21.15.67:0/4228988781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 msgr2=0x5636c07ba820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.407+0000 7efdb9258740 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efda0009970 tx=0x7efda003f000 comp rx=0 tx=0).stop 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 -- 172.21.15.67:0/4228988781 shutdown_connections 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 --2- 172.21.15.67:0/4228988781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07ba820 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 -- 172.21.15.67:0/4228988781 >> 172.21.15.67:0/4228988781 conn(0x5636c07baed0 msgr2=0x5636c07c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 -- 172.21.15.67:0/4228988781 shutdown_connections 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 -- 172.21.15.67:0/4228988781 wait complete. 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 -- start start 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.408+0000 7efdb9258740 1 Processor -- start 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b07f0 con 0x5636c07bd0a0 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b1c60 con 0x5636c07bd9c0 2024-09-10T23:29:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b0ec0 con 0x5636c07c9950 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdb9258740 10 client.0 _ll_register_callbacks cb 0x7efd980099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdaf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38610/0 (socket says 172.21.15.67:38610) 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efdaf400640 1 -- 172.21.15.67:0/723703413 learned_addr learned my addr 172.21.15.67:0/723703413 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.409+0000 7efda6a00640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdafe00640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 -- 172.21.15.67:0/723703413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 msgr2=0x5636c07cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 -- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 msgr2=0x5636c07d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07d2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 -- 172.21.15.67:0/723703413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c07cd280 con 0x5636c07bd0a0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdaf400640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efda0031630 tx=0x7efda0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3d944ac0b1ac8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efd9d600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0031d60 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9519+0+0 (secure 0 0 0) 0x7efda0053a60 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.410+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda0030c30 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efda006b050 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 10 client.16629 fetch_fsmap learned FSMap version 45 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5636c0846d10 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda0030c30 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7efda0043040 con 0x5636c07bd0a0 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 10 client.16629 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 10 client.16629 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:29:07.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 20 client.16629 populate_metadata read hostname 'smithi067' 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c0847420 0x5636c0849840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 4 client.16629 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.411+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5636c07329f0 con 0x5636c0847420 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.412+0000 7efda6a00640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c0847420 0x5636c0849840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.412+0000 7efda6a00640 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c0847420 0x5636c0849840 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7ff10013526164 server_cookie=77c1ec6fda0d6b76 in_seq=0 out_seq=0 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.412+0000 7efd9d600640 10 client.16629 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efd9d600640 1 -- 172.21.15.67:0/723703413 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5636c07329f0 con 0x5636c0847420 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efd9d600640 10 client.16629 handle_command_reply: tid=0 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 client.16629 shutdown 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 msgr2=0x5636c07d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efda0031630 tx=0x7efda0031980 comp rx=0 tx=0).stop 2024-09-10T23:29:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 shutdown_connections 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x5636c0847420 0x5636c0849840 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636c07c9950 0x5636c07cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c07bd9c0 0x5636c07d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 --2- 172.21.15.67:0/723703413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5636c07bd0a0 0x5636c07d1d00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 >> 172.21.15.67:0/723703413 conn(0x5636c07baed0 msgr2=0x5636c07c0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efd9d600640 0 client.16629 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.413+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 shutdown_connections 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb9258740 1 -- 172.21.15.67:0/723703413 wait complete. 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb9258740 20 client.16629 trim_cache size 0 max 16384 2024-09-10T23:29:07.418 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 45, 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 87, 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": {} 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 msgr2=0x7efd940b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 0x7efd940b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efda0006010 tx=0x7efda0031000 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 msgr2=0x7efdb01a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efda800d950 tx=0x7efda800de10 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 shutdown_connections 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7efd940b5450 0x7efd940b7910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdb010b340 0x7efdb01a0ce0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb010a9c0 0x7efdb019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 --2- 172.21.15.67:0/4164862926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efdb0104fb0 0x7efdb019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.414+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 >> 172.21.15.67:0/4164862926 conn(0x7efdb0100a00 msgr2=0x7efdb0101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.415+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 shutdown_connections 2024-09-10T23:29:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.415+0000 7efdb6600640 1 -- 172.21.15.67:0/4164862926 wait complete. 2024-09-10T23:29:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:07 smithi145 ceph-mon[34397]: pgmap v1340: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 54 op/s 2024-09-10T23:29:08.274 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:07 smithi092 ceph-mon[34223]: pgmap v1340: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 54 op/s 2024-09-10T23:29:08.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:07 smithi067 ceph-mon[30939]: pgmap v1340: 129 pgs: 129 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 54 op/s 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.977+0000 7f687aa00640 1 -- 172.21.15.67:0/3169696638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687410f440 msgr2=0x7f6874111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.977+0000 7f687aa00640 1 --2- 172.21.15.67:0/3169696638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687410f440 0x7f6874111830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f686c009ca0 tx=0x7f686c02a950 comp rx=0 tx=0).stop 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.978+0000 7f687aa00640 1 -- 172.21.15.67:0/3169696638 shutdown_connections 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.978+0000 7f687aa00640 1 --2- 172.21.15.67:0/3169696638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687410f440 0x7f6874111830 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.978+0000 7f687aa00640 1 --2- 172.21.15.67:0/3169696638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68741021b0 0x7f687410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.978+0000 7f687aa00640 1 --2- 172.21.15.67:0/3169696638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f6874101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.978+0000 7f687aa00640 1 -- 172.21.15.67:0/3169696638 >> 172.21.15.67:0/3169696638 conn(0x7f68740fd700 msgr2=0x7f68740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.979+0000 7f687aa00640 1 -- 172.21.15.67:0/3169696638 shutdown_connections 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.979+0000 7f687aa00640 1 -- 172.21.15.67:0/3169696638 wait complete. 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.982+0000 7f687aa00640 1 Processor -- start 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.982+0000 7f687aa00640 1 -- start start 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f687410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 0x7f687410bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 0x7f687410c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6874114e70 con 0x7f687410f440 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6874114cf0 con 0x7f6874101890 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.983+0000 7f687aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6874114ff0 con 0x7f68741021b0 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.985+0000 7f6873e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f687410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:08.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.985+0000 7f6878a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 0x7f687410c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.985+0000 7f6873e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f687410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:54444/0 (socket says 172.21.15.67:54444) 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.985+0000 7f6873e00640 1 -- 172.21.15.67:0/321451760 learned_addr learned my addr 172.21.15.67:0/321451760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.985+0000 7f686be00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 0x7f687410bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.986+0000 7f6878a00640 1 -- 172.21.15.67:0/321451760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 msgr2=0x7f687410bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.986+0000 7f6878a00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 0x7f687410bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.986+0000 7f6878a00640 1 -- 172.21.15.67:0/321451760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 msgr2=0x7f687410ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.986+0000 7f6878a00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f687410ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.986+0000 7f6878a00640 1 -- 172.21.15.67:0/321451760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687410cd40 con 0x7f687410f440 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.987+0000 7f686be00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 0x7f687410bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.987+0000 7f6878a00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 0x7f687410c480 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f686c009bf0 tx=0x7f686c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.988+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c034040 con 0x7f687410f440 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.988+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f686c02fcf0 con 0x7f687410f440 2024-09-10T23:29:08.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.988+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c004530 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.988+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68741abc70 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.989+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68741ac100 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:07.991+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6874101c70 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.005+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f686c0046d0 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.008+0000 7f6871600640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 0x7f68580b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.008+0000 7f6873e00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 0x7f68580b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.009+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f686c107900 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.010+0000 7f6873e00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 0x7f68580b7b10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f685c009a40 tx=0x7f685c0091c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.010+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f686c0ce0f0 con 0x7f687410f440 2024-09-10T23:29:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.412+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f687410d200 con 0x7f687410f440 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"btime":"2024-09-10T23:28:38:783141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:28:38.628606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.413+0000 7f6871600640 1 -- 172.21.15.67:0/321451760 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11910 (secure 0 0 0) 0x7f686c0cd9f0 con 0x7f687410f440 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 msgr2=0x7f68580b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 0x7f68580b7b10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f685c009a40 tx=0x7f685c0091c0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 msgr2=0x7f687410c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 0x7f687410c480 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f686c009bf0 tx=0x7f686c003f10 comp rx=0 tx=0).stop 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 shutdown_connections 2024-09-10T23:29:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f68580b5650 0x7f68580b7b10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f687410f440 0x7f687410c480 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68741021b0 0x7f687410bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 --2- 172.21.15.67:0/321451760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6874101890 0x7f687410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.415+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 >> 172.21.15.67:0/321451760 conn(0x7f68740fd700 msgr2=0x7f68740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.416+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 shutdown_connections 2024-09-10T23:29:08.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:08.416+0000 7f687aa00640 1 -- 172.21.15.67:0/321451760 wait complete. 2024-09-10T23:29:09.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:08 smithi145 ceph-mon[34397]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T23:29:09.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:08 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/321451760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:29:09.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:08 smithi092 ceph-mon[34223]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T23:29:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:08 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/321451760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:29:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:08 smithi067 ceph-mon[30939]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T23:29:09.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:08 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/321451760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:29:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:09 smithi145 ceph-mon[34397]: pgmap v1342: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:09 smithi092 ceph-mon[34223]: pgmap v1342: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:10.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:09 smithi067 ceph-mon[30939]: pgmap v1342: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:11.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:10 smithi145 ceph-mon[34397]: pgmap v1343: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:11.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:10 smithi092 ceph-mon[34223]: pgmap v1343: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:11.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:10 smithi067 ceph-mon[30939]: pgmap v1343: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-10T23:29:11.897 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34730 quiesce db --set-id eff1017 --release --await' 2024-09-10T23:29:11.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34730 quiesce db --set-id eff1017 --release --await 2024-09-10T23:29:12.660 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:14.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:13 smithi145 ceph-mon[34397]: pgmap v1344: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-10T23:29:14.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:13 smithi092 ceph-mon[34223]: pgmap v1344: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-10T23:29:14.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:13 smithi067 ceph-mon[30939]: pgmap v1344: 129 pgs: 17 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 63 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-09-10T23:29:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 -- 172.21.15.67:0/3208264187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c9010a9b0 msgr2=0x7f7c9010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 --2- 172.21.15.67:0/3208264187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c9010a9b0 0x7f7c9010adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009860 tx=0x7f7c7c02a340 comp rx=0 tx=0).stop 2024-09-10T23:29:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 -- 172.21.15.67:0/3208264187 shutdown_connections 2024-09-10T23:29:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 --2- 172.21.15.67:0/3208264187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c9010b330 0x7f7c90111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 --2- 172.21.15.67:0/3208264187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c9010a9b0 0x7f7c9010adf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 --2- 172.21.15.67:0/3208264187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90104fa0 0x7f7c90105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 -- 172.21.15.67:0/3208264187 >> 172.21.15.67:0/3208264187 conn(0x7f7c90100a50 msgr2=0x7f7c90102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 -- 172.21.15.67:0/3208264187 shutdown_connections 2024-09-10T23:29:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.983+0000 7f7c94c00640 1 -- 172.21.15.67:0/3208264187 wait complete. 2024-09-10T23:29:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 Processor -- start 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 -- start start 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c90104fa0 0x7f7c900776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c9010a9b0 0x7f7c90077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c9010b330 0x7f7c90078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90114de0 con 0x7f7c9010a9b0 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90114c60 con 0x7f7c9010b330 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.985+0000 7f7c94c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90114f60 con 0x7f7c90104fa0 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c9010b330 0x7f7c90078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c9010b330 0x7f7c90078130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:36746/0 (socket says 172.21.15.67:36746) 2024-09-10T23:29:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8e800640 1 -- 172.21.15.67:0/3040158419 learned_addr learned my addr 172.21.15.67:0/3040158419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8de00640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c90104fa0 0x7f7c900776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8d400640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c9010a9b0 0x7f7c90077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.986+0000 7f7c8e800640 1 -- 172.21.15.67:0/3040158419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c90104fa0 msgr2=0x7f7c900776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c8e800640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c90104fa0 0x7f7c900776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c8e800640 1 -- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c9010a9b0 msgr2=0x7f7c90077bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c8e800640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c9010a9b0 0x7f7c90077bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c8e800640 1 -- 172.21.15.67:0/3040158419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9019b140 con 0x7f7c9010b330 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c8e800640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c9010b330 0x7f7c90078130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800efd0 tx=0x7f7c8800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c86a00640 1 -- 172.21.15.67:0/3040158419 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88003b70 con 0x7f7c9010b330 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c86a00640 1 -- 172.21.15.67:0/3040158419 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c88003d10 con 0x7f7c9010b330 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c94c00640 1 -- 172.21.15.67:0/3040158419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9019b430 con 0x7f7c9010b330 2024-09-10T23:29:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.987+0000 7f7c86a00640 1 -- 172.21.15.67:0/3040158419 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c880107a0 con 0x7f7c9010b330 2024-09-10T23:29:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.988+0000 7f7c94c00640 1 -- 172.21.15.67:0/3040158419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9019b940 con 0x7f7c9010b330 2024-09-10T23:29:14.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.991+0000 7f7c86a00640 1 -- 172.21.15.67:0/3040158419 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7c8801f020 con 0x7f7c9010b330 2024-09-10T23:29:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.993+0000 7f7c86a00640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c740b54d0 0x7f7c740b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.993+0000 7f7c86a00640 1 -- 172.21.15.67:0/3040158419 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f7c880e65c0 con 0x7f7c9010b330 2024-09-10T23:29:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.994+0000 7f7c8de00640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c740b54d0 0x7f7c740b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:14.995+0000 7f7c8de00640 1 --2- 172.21.15.67:0/3040158419 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c740b54d0 0x7f7c740b7990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005fd0 tx=0x7f7c80009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:15.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.005+0000 7f7c97643740 1 Processor -- start 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.005+0000 7f7c97643740 1 -- start start 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 0x56312fe44c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56312fdad290 con 0x56312fe428a0 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56312fd9c6c0 con 0x56312fe36910 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.006+0000 7f7c97643740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56312fda04d0 con 0x56312fe35ff0 2024-09-10T23:29:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57082/0 (socket says 172.21.15.67:57082) 2024-09-10T23:29:15.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.67:36748/0 (socket says 172.21.15.67:36748) 2024-09-10T23:29:15.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8de00640 1 -- 172.21.15.67:0/666649647 learned_addr learned my addr 172.21.15.67:0/666649647 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8d400640 1 -- 172.21.15.67:0/666649647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 msgr2=0x56312fe363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8d400640 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8e800640 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 0x56312fe44c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.007+0000 7f7c8d400640 1 -- 172.21.15.67:0/666649647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 msgr2=0x56312fe44c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.008+0000 7f7c8d400640 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 0x56312fe44c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.008+0000 7f7c8d400640 1 -- 172.21.15.67:0/666649647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312fd29460 con 0x56312fe36910 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.008+0000 7f7c8e800640 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 0x56312fe44c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.008+0000 7f7c8d400640 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a860 tx=0x7f7c7c0094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a452e2abedeec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.009+0000 7f7c7b000640 1 -- 172.21.15.67:0/666649647 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c043070 con 0x56312fe36910 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.009+0000 7f7c7b000640 1 -- 172.21.15.67:0/666649647 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c7c0044d0 con 0x56312fe36910 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.009+0000 7f7c7b000640 1 -- 172.21.15.67:0/666649647 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c037040 con 0x56312fe36910 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.010+0000 7f7c97643740 1 -- 172.21.15.67:0/666649647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 msgr2=0x56312fe42360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.010+0000 7f7c97643740 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a860 tx=0x7f7c7c0094f0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 -- 172.21.15.67:0/666649647 shutdown_connections 2024-09-10T23:29:15.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe428a0 0x56312fe44c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe36910 0x56312fe42360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 --2- 172.21.15.67:0/666649647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 -- 172.21.15.67:0/666649647 >> 172.21.15.67:0/666649647 conn(0x56312fe33e20 msgr2=0x56312fe34fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 -- 172.21.15.67:0/666649647 shutdown_connections 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.011+0000 7f7c97643740 1 -- 172.21.15.67:0/666649647 wait complete. 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.012+0000 7f7c97643740 1 -- start start 2024-09-10T23:29:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.013+0000 7f7c97643740 1 Processor -- start 2024-09-10T23:29:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56312fbfdbb0 con 0x56312fe40600 2024-09-10T23:29:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56312fd376c0 con 0x56312fe428a0 2024-09-10T23:29:15.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56312fe56500 con 0x56312fe35ff0 2024-09-10T23:29:15.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c97643740 10 client.0 _ll_register_callbacks cb 0x56312fd52490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:29:15.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.014+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.015+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57086/0 (socket says 172.21.15.67:57086) 2024-09-10T23:29:15.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.015+0000 7f7c8de00640 1 -- 172.21.15.67:0/2197741025 learned_addr learned my addr 172.21.15.67:0/2197741025 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:15.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8e800640 1 -- 172.21.15.67:0/2197741025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 msgr2=0x56312fe3fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8e800640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8e800640 1 -- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 msgr2=0x56312fe40070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8d400640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8e800640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe40070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.016+0000 7f7c8e800640 1 -- 172.21.15.67:0/2197741025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312fafab00 con 0x56312fe40600 2024-09-10T23:29:15.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.019+0000 7f7c8de00640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:29:15.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.019+0000 7f7c8d400640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:15.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.020+0000 7f7c8e800640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7c880e3870 tx=0x7f7c880acb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c24420db30ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:15.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.020+0000 7f7c6f400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-09-10T23:29:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.022+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c880abd10 con 0x56312fe40600 2024-09-10T23:29:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.022+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f7c880a7ef0 con 0x56312fe40600 2024-09-10T23:29:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.022+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c7c0044d0 con 0x56312fe40600 2024-09-10T23:29:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.023+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c880c4050 con 0x56312fe40600 2024-09-10T23:29:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.023+0000 7f7c97643740 10 client.16641 fetch_fsmap learned FSMap version 45 2024-09-10T23:29:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.023+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56312fb71960 con 0x56312fe40600 2024-09-10T23:29:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.024+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c7c0044d0 con 0x56312fe40600 2024-09-10T23:29:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.024+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f7c880c7080 con 0x56312fe40600 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 10 client.16641 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 10 client.16641 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 20 client.16641 populate_metadata read hostname 'smithi067' 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fe63960 0x56312fe65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 4 client.16641 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.025+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312fdaf1e0 con 0x56312fe63960 2024-09-10T23:29:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.026+0000 7f7c8de00640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fe63960 0x56312fe65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.029+0000 7f7c8de00640 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fe63960 0x56312fe65dc0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577718a8ba0cfc2b server_cookie=1e95255ba93b419e in_seq=0 out_seq=0 2024-09-10T23:29:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.029+0000 7f7c6f400640 10 client.16641 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.030+0000 7f7c6f400640 1 -- 172.21.15.67:0/2197741025 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56312fdaf1e0 con 0x56312fe63960 2024-09-10T23:29:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.030+0000 7f7c6f400640 10 client.16641 handle_command_reply: tid=0 2024-09-10T23:29:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.030+0000 7f7c97643740 1 client.16641 shutdown 2024-09-10T23:29:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.031+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 msgr2=0x56312fe5f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.031+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7c880e3870 tx=0x7f7c880acb70 comp rx=0 tx=0).stop 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.031+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 shutdown_connections 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.031+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fe63960 0x56312fe65dc0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c6f400640 0 client.16641 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 --2- 172.21.15.67:0/2197741025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 >> 172.21.15.67:0/2197741025 conn(0x56312fe33e20 msgr2=0x56312fe399c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 shutdown_connections 2024-09-10T23:29:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 1 -- 172.21.15.67:0/2197741025 wait complete. 2024-09-10T23:29:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.032+0000 7f7c97643740 20 client.16641 trim_cache size 0 max 16384 2024-09-10T23:29:15.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.057+0000 7f7c97643740 1 Processor -- start 2024-09-10T23:29:15.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.057+0000 7f7c97643740 1 -- start start 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c900718f0 con 0x56312fe40600 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006f130 con 0x56312fe428a0 2024-09-10T23:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c97643740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006cfc0 con 0x56312fe35ff0 2024-09-10T23:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57088/0 (socket says 172.21.15.67:57088) 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c8de00640 1 -- 172.21.15.67:0/234004332 learned_addr learned my addr 172.21.15.67:0/234004332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.058+0000 7f7c8d400640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8e800640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 -- 172.21.15.67:0/234004332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 msgr2=0x56312fe3eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe3eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 -- 172.21.15.67:0/234004332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 msgr2=0x56312fe5f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 -- 172.21.15.67:0/234004332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312fe5fc70 con 0x56312fe35ff0 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8e800640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8d400640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c8de00640 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c80003980 tx=0x7f7c80014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b35a00d4e699727 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.059+0000 7f7c7a600640 1 -- 172.21.15.67:0/234004332 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80020110 con 0x56312fe35ff0 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.060+0000 7f7c7a600640 1 -- 172.21.15.67:0/234004332 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c80020da0 con 0x56312fe35ff0 2024-09-10T23:29:15.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.060+0000 7f7c7a600640 1 -- 172.21.15.67:0/234004332 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800047a0 con 0x56312fe35ff0 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 -- 172.21.15.67:0/234004332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 msgr2=0x56312fe3fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c80003980 tx=0x7f7c80014000 comp rx=0 tx=0).stop 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 -- 172.21.15.67:0/234004332 shutdown_connections 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe5f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 --2- 172.21.15.67:0/234004332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe3fcd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 -- 172.21.15.67:0/234004332 >> 172.21.15.67:0/234004332 conn(0x56312fe33e20 msgr2=0x56312fe3e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 -- 172.21.15.67:0/234004332 shutdown_connections 2024-09-10T23:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.061+0000 7f7c97643740 1 -- 172.21.15.67:0/234004332 wait complete. 2024-09-10T23:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.062+0000 7f7c97643740 1 -- start start 2024-09-10T23:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.062+0000 7f7c97643740 1 Processor -- start 2024-09-10T23:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe58e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006d610 con 0x56312fe40600 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006ea60 con 0x56312fe428a0 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9006dcc0 con 0x56312fe35ff0 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c97643740 10 client.0 _ll_register_callbacks cb 0x7f7c88009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c8e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe58e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.67:57090/0 (socket says 172.21.15.67:57090) 2024-09-10T23:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.063+0000 7f7c8de00640 1 -- 172.21.15.67:0/1928154528 learned_addr learned my addr 172.21.15.67:0/1928154528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8d400640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 -- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 msgr2=0x56312fe58e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe58e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 -- 172.21.15.67:0/1928154528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 msgr2=0x56312fe586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 -- 172.21.15.67:0/1928154528 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312fed08a0 con 0x56312fe35ff0 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.064+0000 7f7c8de00640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005890 tx=0x7f7c800202f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30682b76cc5b2a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.065+0000 7f7c6f400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-10T23:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.065+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8002d040 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.066+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c80020da0 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.065+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f7c8002a630 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.066+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c80044050 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.066+0000 7f7c97643740 10 client.26350 fetch_fsmap learned FSMap version 45 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.066+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56312fed1b50 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c80020da0 con 0x56312fe35ff0 2024-09-10T23:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f7c8003b960 con 0x56312fe35ff0 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 10 client.26350 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 10 client.26350 resolve_mds: validated gid 34730 aka daemon mds.b 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 20 client.26350 populate_metadata read hostname 'smithi067' 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fed2270 0x56312fed4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 4 client.26350 mds_command: new command op to 34730 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eff1017", "release": true, "await": true}] 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.067+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 --> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eff1017", "release": true, "await": true}) -- 0x56312fdaac90 con 0x56312fed2270 2024-09-10T23:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.068+0000 7f7c8d400640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fed2270 0x56312fed4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:15.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.070+0000 7f7c8d400640 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fed2270 0x56312fed4690 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415096991e5d1cd2 server_cookie=9dc20ff0e8435c07 in_seq=0 out_seq=0 2024-09-10T23:29:15.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.070+0000 7f7c6f400640 10 client.26350 ms_handle_connect on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.071+0000 7f7c6f400640 1 -- 172.21.15.67:0/1928154528 <== mds.0 v2:172.21.15.92:6838/1949986898 1 ==== command_reply(tid 0: -1 ) ==== 8+0+590 (crc 0 0 0) 0x56312fdaac90 con 0x56312fed2270 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.071+0000 7f7c6f400640 10 client.26350 handle_command_reply: tid=0 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.071+0000 7f7c97643740 1 client.26350 shutdown 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.071+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 msgr2=0x56312fe5c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.071+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005890 tx=0x7f7c800202f0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 shutdown_connections 2024-09-10T23:29:15.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:6838/1949986898,v1:172.21.15.92:6839/1949986898] conn(0x56312fed2270 0x56312fed4690 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56312fe428a0 0x56312fe58e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56312fe40600 0x56312fe586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 --2- 172.21.15.67:0/1928154528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56312fe35ff0 0x56312fe5c1c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 >> 172.21.15.67:0/1928154528 conn(0x56312fe33e20 msgr2=0x56312fe3ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.072+0000 7f7c6f400640 0 client.26350 ms_handle_reset on v2:172.21.15.92:6838/1949986898 2024-09-10T23:29:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.073+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 shutdown_connections 2024-09-10T23:29:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.073+0000 7f7c97643740 1 -- 172.21.15.67:0/1928154528 wait complete. 2024-09-10T23:29:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:15.073+0000 7f7c97643740 20 client.26350 trim_cache size 0 max 16384 2024-09-10T23:29:15.075 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 45, 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "leader": 34730, 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "set_version": 87, 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "sets": { 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "eff1017": { 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "version": 87, 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "age_ref": 7.7, 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "CANCELED", 2024-09-10T23:29:15.076 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "timeout": 90.0, 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "expiration": 148.4, 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "members": { 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "file:/": { 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "excluded": true, 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "state": { 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "QUIESCED", 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 24.6 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:15.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:15.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:15.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:15.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:15.078 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:Error EPERM: 2024-09-10T23:29:16.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:15 smithi145 ceph-mon[34397]: pgmap v1345: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T23:29:16.238 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:15 smithi067 ceph-mon[30939]: pgmap v1345: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T23:29:16.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:15 smithi092 ceph-mon[34223]: pgmap v1345: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T23:29:16.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:29:16.602 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T23:29:16.602 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-10T23:29:16.612 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-10T23:29:16.612 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T23:29:16.612 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T23:29:16.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-10T23:29:16.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-10T23:29:16.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T23:29:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 -- 172.21.15.67:0/2666616594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b8075e00 msgr2=0x7f58b810eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 --2- 172.21.15.67:0/2666616594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b8075e00 0x7f58b810eba0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009900 tx=0x7f58a402a410 comp rx=0 tx=0).stop 2024-09-10T23:29:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 -- 172.21.15.67:0/2666616594 shutdown_connections 2024-09-10T23:29:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 --2- 172.21.15.67:0/2666616594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b810f0e0 0x7f58b81114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 --2- 172.21.15.67:0/2666616594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b8075e00 0x7f58b810eba0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 --2- 172.21.15.67:0/2666616594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b80758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 -- 172.21.15.67:0/2666616594 >> 172.21.15.67:0/2666616594 conn(0x7f58b80fe580 msgr2=0x7f58b81009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 -- 172.21.15.67:0/2666616594 shutdown_connections 2024-09-10T23:29:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.088+0000 7f58bec00640 1 -- 172.21.15.67:0/2666616594 wait complete. 2024-09-10T23:29:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.089+0000 7f58bec00640 1 Processor -- start 2024-09-10T23:29:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.089+0000 7f58bec00640 1 -- start start 2024-09-10T23:29:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.089+0000 7f58bec00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b8075e00 0x7f58b819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8114bc0 con 0x7f58b810f0e0 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bec00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8114a40 con 0x7f58b80754e0 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8114d40 con 0x7f58b8075e00 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34656/0 (socket says 172.21.15.67:34656) 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 -- 172.21.15.67:0/2413640388 learned_addr learned my addr 172.21.15.67:0/2413640388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bd800640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bce00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b8075e00 0x7f58b819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 -- 172.21.15.67:0/2413640388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b8075e00 msgr2=0x7f58b819b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b8075e00 0x7f58b819b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 -- 172.21.15.67:0/2413640388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 msgr2=0x7f58b819b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b819b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 -- 172.21.15.67:0/2413640388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b81a12c0 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58bd800640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b819b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.090+0000 7f58be200640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009fd0 tx=0x7f58b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.091+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0017070 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.091+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b81a15b0 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.091+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58b0003e00 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.091+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0010460 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.091+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b81a7c70 con 0x7f58b810f0e0 2024-09-10T23:29:17.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.092+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b80758c0 con 0x7f58b810f0e0 2024-09-10T23:29:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.094+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f58b0003970 con 0x7f58b810f0e0 2024-09-10T23:29:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.096+0000 7f58b6000640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 0x7f58a00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.096+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f58b00e6600 con 0x7f58b810f0e0 2024-09-10T23:29:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.096+0000 7f58bd800640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 0x7f58a00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.097+0000 7f58bd800640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 0x7f58a00b7910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0058c0 tx=0x7f58ac005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.100+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58b00acd40 con 0x7f58b810f0e0 2024-09-10T23:29:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.313+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f58b8063590 con 0x7f58a00b5450 2024-09-10T23:29:17.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.323+0000 7f58b6000640 1 -- 172.21.15.67:0/2413640388 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f58b8063590 con 0x7f58a00b5450 2024-09-10T23:29:17.325 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T22:59:24", "first": "2024-09-10T23:00:00", "last": "2024-09-10T23:29:00", "last_pruned": "2024-09-10T23:29:01", "created_count": 30, "pruned_count": 24, "active": true} 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 msgr2=0x7f58a00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 0x7f58a00b7910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0058c0 tx=0x7f58ac005850 comp rx=0 tx=0).stop 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 msgr2=0x7f58b81a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009fd0 tx=0x7f58b000b260 comp rx=0 tx=0).stop 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 shutdown_connections 2024-09-10T23:29:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f58a00b5450 0x7f58a00b7910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58b810f0e0 0x7f58b81a0af0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b8075e00 0x7f58b819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 --2- 172.21.15.67:0/2413640388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80754e0 0x7f58b819b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 >> 172.21.15.67:0/2413640388 conn(0x7f58b80fe580 msgr2=0x7f58b80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 shutdown_connections 2024-09-10T23:29:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.326+0000 7f58bec00640 1 -- 172.21.15.67:0/2413640388 wait complete. 2024-09-10T23:29:17.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 -- 172.21.15.67:0/2957617678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010a720 msgr2=0x7f2f7010ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 --2- 172.21.15.67:0/2957617678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010a720 0x7f2f7010ab60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009900 tx=0x7f2f5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 -- 172.21.15.67:0/2957617678 shutdown_connections 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 --2- 172.21.15.67:0/2957617678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f7010b0a0 0x7f2f701118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 --2- 172.21.15.67:0/2957617678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010a720 0x7f2f7010ab60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 --2- 172.21.15.67:0/2957617678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f70104d20 0x7f2f70105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 -- 172.21.15.67:0/2957617678 >> 172.21.15.67:0/2957617678 conn(0x7f2f70100740 msgr2=0x7f2f70102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 -- 172.21.15.67:0/2957617678 shutdown_connections 2024-09-10T23:29:17.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.503+0000 7f2f6f200640 1 -- 172.21.15.67:0/2957617678 wait complete. 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.504+0000 7f2f6f200640 1 Processor -- start 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.504+0000 7f2f6f200640 1 -- start start 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 0x7f2f7019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7010a720 0x7f2f7019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70114ba0 con 0x7f2f7010b0a0 2024-09-10T23:29:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70114a20 con 0x7f2f7010a720 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70114d20 con 0x7f2f70104d20 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34672/0 (socket says 172.21.15.67:34672) 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 -- 172.21.15.67:0/961325728 learned_addr learned my addr 172.21.15.67:0/961325728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6d400640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7010a720 0x7f2f7019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6de00640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 0x7f2f7019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 -- 172.21.15.67:0/961325728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 msgr2=0x7f2f7019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 0x7f2f7019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 -- 172.21.15.67:0/961325728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7010a720 msgr2=0x7f2f7019b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7010a720 0x7f2f7019b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6e800640 1 -- 172.21.15.67:0/961325728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701a1240 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.505+0000 7f2f6de00640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 0x7f2f7019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f6e800640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009fd0 tx=0x7f2f6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68017070 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701a1530 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f68003e00 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68010460 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.506+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701a7bf0 con 0x7f2f7010b0a0 2024-09-10T23:29:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.507+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f70105100 con 0x7f2f7010b0a0 2024-09-10T23:29:17.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.510+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2f68003970 con 0x7f2f7010b0a0 2024-09-10T23:29:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.511+0000 7f2f66a00640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 0x7f2f540b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.511+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f2f680e6690 con 0x7f2f7010b0a0 2024-09-10T23:29:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.511+0000 7f2f6de00640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 0x7f2f540b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:17.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.512+0000 7f2f6de00640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 0x7f2f540b7910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009b40 tx=0x7f2f60005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:17.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.513+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f680acdd0 con 0x7f2f7010b0a0 2024-09-10T23:29:17.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.730+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 --> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2f70063590 con 0x7f2f540b5450 2024-09-10T23:29:17.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.739+0000 7f2f66a00640 1 -- 172.21.15.67:0/961325728 <== mgr.14223 v2:172.21.15.67:6800/1232465191 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2f70063590 con 0x7f2f540b5450 2024-09-10T23:29:17.740 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 msgr2=0x7f2f540b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 0x7f2f540b7910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009b40 tx=0x7f2f60005e50 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 msgr2=0x7f2f701a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009fd0 tx=0x7f2f6800b260 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 shutdown_connections 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f2f540b5450 0x7f2f540b7910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f7010b0a0 0x7f2f701a0a70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7010a720 0x7f2f7019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 --2- 172.21.15.67:0/961325728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f70104d20 0x7f2f7019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.741+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 >> 172.21.15.67:0/961325728 conn(0x7f2f70100740 msgr2=0x7f2f70102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.742+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 shutdown_connections 2024-09-10T23:29:17.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:17.742+0000 7f2f6f200640 1 -- 172.21.15.67:0/961325728 wait complete. 2024-09-10T23:29:17.759 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T23:29:18.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:29:18.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:17 smithi067 ceph-mon[30939]: pgmap v1346: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 306 B/s wr, 6 op/s 2024-09-10T23:29:18.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:17 smithi145 ceph-mon[34397]: pgmap v1346: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 306 B/s wr, 6 op/s 2024-09-10T23:29:18.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:17 smithi092 ceph-mon[34223]: pgmap v1346: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 306 B/s wr, 6 op/s 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 -- 172.21.15.67:0/4292268151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 msgr2=0x7fd9b41052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/4292268151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b41052e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009900 tx=0x7fd9a402a400 comp rx=0 tx=0).stop 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 -- 172.21.15.67:0/4292268151 shutdown_connections 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/4292268151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 0x7fd9b4111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/4292268151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 0x7fd9b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/4292268151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b41052e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.287+0000 7fd9b9e00640 1 -- 172.21.15.67:0/4292268151 >> 172.21.15.67:0/4292268151 conn(0x7fd9b4100920 msgr2=0x7fd9b4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:18.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.288+0000 7fd9b9e00640 1 -- 172.21.15.67:0/4292268151 shutdown_connections 2024-09-10T23:29:18.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.288+0000 7fd9b9e00640 1 -- 172.21.15.67:0/4292268151 wait complete. 2024-09-10T23:29:18.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.288+0000 7fd9b9e00640 1 Processor -- start 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 -- start start 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 0x7fd9b419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 0x7fd9b41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4114da0 con 0x7fd9b4104f00 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4114c20 con 0x7fd9b410b280 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4114f20 con 0x7fd9b410a900 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34694/0 (socket says 172.21.15.67:34694) 2024-09-10T23:29:18.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b2a00640 1 -- 172.21.15.67:0/202184259 learned_addr learned my addr 172.21.15.67:0/202184259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b3400640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 0x7fd9b41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9aa000640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 0x7fd9b419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.289+0000 7fd9b2a00640 1 -- 172.21.15.67:0/202184259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 msgr2=0x7fd9b419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b2a00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 0x7fd9b419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b2a00640 1 -- 172.21.15.67:0/202184259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 msgr2=0x7fd9b41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b2a00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 0x7fd9b41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b2a00640 1 -- 172.21.15.67:0/202184259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b41a1320 con 0x7fd9b4104f00 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b2a00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a920 tx=0x7fd9a4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a403f070 con 0x7fd9b4104f00 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9a403bdc0 con 0x7fd9b4104f00 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b41a15b0 con 0x7fd9b4104f00 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4035030 con 0x7fd9b4104f00 2024-09-10T23:29:18.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.290+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a7cd0 con 0x7fd9b4104f00 2024-09-10T23:29:18.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.291+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b41052e0 con 0x7fd9b4104f00 2024-09-10T23:29:18.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.295+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd9a4031500 con 0x7fd9b4104f00 2024-09-10T23:29:18.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.296+0000 7fd9b8a00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 0x7fd9980b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.297+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7fd9a4107570 con 0x7fd9b4104f00 2024-09-10T23:29:18.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.297+0000 7fd9aa000640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 0x7fd9980b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.297+0000 7fd9aa000640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 0x7fd9980b7910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e10 tx=0x7fd99c005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:18.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.298+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd9a40cdce0 con 0x7fd9b4104f00 2024-09-10T23:29:18.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.497+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9b41a8000 con 0x7fd9b4104f00 2024-09-10T23:29:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.499+0000 7fd9b8a00640 1 -- 172.21.15.67:0/202184259 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+25546 (secure 0 0 0) 0x7fd9a40cd5e0 con 0x7fd9b4104f00 2024-09-10T23:29:18.501 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:29:18.501 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":136,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T23:29:07.835412+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]},{"pool":4,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]},{"pool":4,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:29:18.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 msgr2=0x7fd9980b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 0x7fd9980b7910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e10 tx=0x7fd99c005da0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 msgr2=0x7fd9b419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a920 tx=0x7fd9a4003d60 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 shutdown_connections 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fd9980b5450 0x7fd9980b7910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b410b280 0x7fd9b41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b410a900 0x7fd9b419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 --2- 172.21.15.67:0/202184259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9b4104f00 0x7fd9b419b280 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 >> 172.21.15.67:0/202184259 conn(0x7fd9b4100920 msgr2=0x7fd9b4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 shutdown_connections 2024-09-10T23:29:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.501+0000 7fd9b9e00640 1 -- 172.21.15.67:0/202184259 wait complete. 2024-09-10T23:29:18.519 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:50:44.441563+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:56:14.927478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:56:17.858506+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 136, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T22:56:25.944001+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 136, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T23:29:18.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:29:18.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 -- 172.21.15.67:0/3795002231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c410f2b0 msgr2=0x7f78c41116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 --2- 172.21.15.67:0/3795002231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c410f2b0 0x7f78c41116a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009ca0 tx=0x7f78bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 -- 172.21.15.67:0/3795002231 shutdown_connections 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 --2- 172.21.15.67:0/3795002231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c410f2b0 0x7f78c41116a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 --2- 172.21.15.67:0/3795002231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 0x7f78c410ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 --2- 172.21.15.67:0/3795002231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c40754e0 0x7f78c40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 -- 172.21.15.67:0/3795002231 >> 172.21.15.67:0/3795002231 conn(0x7f78c40fe730 msgr2=0x7f78c4100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 -- 172.21.15.67:0/3795002231 shutdown_connections 2024-09-10T23:29:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.681+0000 7f78c9a00640 1 -- 172.21.15.67:0/3795002231 wait complete. 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.682+0000 7f78c9a00640 1 Processor -- start 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.682+0000 7f78c9a00640 1 -- start start 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.682+0000 7f78c9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 0x7f78c419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c9a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 0x7f78c41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4114d90 con 0x7f78c40754e0 2024-09-10T23:29:18.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c9a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4114c10 con 0x7f78c410f2b0 2024-09-10T23:29:18.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4114f10 con 0x7f78c4075e00 2024-09-10T23:29:18.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34708/0 (socket says 172.21.15.67:34708) 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 -- 172.21.15.67:0/994722224 learned_addr learned my addr 172.21.15.67:0/994722224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2000640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 0x7f78c419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c3400640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 0x7f78c41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 -- 172.21.15.67:0/994722224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 msgr2=0x7f78c419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 0x7f78c419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 -- 172.21.15.67:0/994722224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 msgr2=0x7f78c41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 0x7f78c41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c2a00640 1 -- 172.21.15.67:0/994722224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c41a13b0 con 0x7f78c40754e0 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.683+0000 7f78c3400640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 0x7f78c41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:29:18.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78c2a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c40754e0 2024-09-10T23:29:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78b4009c30 con 0x7f78c40754e0 2024-09-10T23:29:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b400f620 con 0x7f78c40754e0 2024-09-10T23:29:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c41a16a0 con 0x7f78c40754e0 2024-09-10T23:29:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.684+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c41a7f40 con 0x7f78c40754e0 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.685+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c40769f0 con 0x7f78c40754e0 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.687+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f78b4010460 con 0x7f78c40754e0 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.689+0000 7f78bb400640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 0x7f78a80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.689+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9607+0+0 (secure 0 0 0) 0x7f78b40e6060 con 0x7f78c40754e0 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.689+0000 7f78c2000640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 0x7f78a80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.690+0000 7f78c2000640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 0x7f78a80b7990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78c41100c0 tx=0x7f78b0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:18.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.692+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78b40ac740 con 0x7f78c40754e0 2024-09-10T23:29:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.892+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78c41a81f0 con 0x7f78c40754e0 2024-09-10T23:29:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.893+0000 7f78bb400640 1 -- 172.21.15.67:0/994722224 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7f78b4009de0 con 0x7f78c40754e0 2024-09-10T23:29:18.894 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.895+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 msgr2=0x7f78a80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.895+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 0x7f78a80b7990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78c41100c0 tx=0x7f78b0005f90 comp rx=0 tx=0).stop 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.895+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 msgr2=0x7f78c419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.895+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400de10 comp rx=0 tx=0).stop 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 shutdown_connections 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78a80b54d0 0x7f78a80b7990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c410f2b0 0x7f78c41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78c4075e00 0x7f78c419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 --2- 172.21.15.67:0/994722224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78c40754e0 0x7f78c419b310 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 >> 172.21.15.67:0/994722224 conn(0x7f78c40fe730 msgr2=0x7f78c4111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 shutdown_connections 2024-09-10T23:29:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:18.896+0000 7f78c9a00640 1 -- 172.21.15.67:0/994722224 wait complete. 2024-09-10T23:29:18.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.075+0000 7fed02200640 1 -- 172.21.15.67:0/599815868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc1086d0 msgr2=0x7fecfc108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.075+0000 7fed02200640 1 --2- 172.21.15.67:0/599815868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc1086d0 0x7fecfc108b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fece4009900 tx=0x7fece402a410 comp rx=0 tx=0).stop 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 -- 172.21.15.67:0/599815868 shutdown_connections 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 --2- 172.21.15.67:0/599815868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc109050 0x7fecfc10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 --2- 172.21.15.67:0/599815868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc1086d0 0x7fecfc108b10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 --2- 172.21.15.67:0/599815868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 0x7fecfc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 -- 172.21.15.67:0/599815868 >> 172.21.15.67:0/599815868 conn(0x7fecfc0fe710 msgr2=0x7fecfc100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:19.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 -- 172.21.15.67:0/599815868 shutdown_connections 2024-09-10T23:29:19.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 -- 172.21.15.67:0/599815868 wait complete. 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.076+0000 7fed02200640 1 Processor -- start 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 -- start start 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 0x7fecfc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 0x7fecfc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc112b80 con 0x7fecfc109050 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc112a00 con 0x7fecfc102cd0 2024-09-10T23:29:19.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fed02200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc112d00 con 0x7fecfc1086d0 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58810/0 (socket says 172.21.15.67:58810) 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fecfbe00640 1 -- 172.21.15.67:0/4098679282 learned_addr learned my addr 172.21.15.67:0/4098679282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fecfb400640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 0x7fecfc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.077+0000 7fecf2a00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 0x7fecfc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 -- 172.21.15.67:0/4098679282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 msgr2=0x7fecfc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 0x7fecfc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 -- 172.21.15.67:0/4098679282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 msgr2=0x7fecfc199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 0x7fecfc199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 -- 172.21.15.67:0/4098679282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc19f1e0 con 0x7fecfc109050 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecf2a00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 0x7fecfc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecfbe00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fecf400eaa0 tx=0x7fecf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400cdb0 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf4005020 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc19f4d0 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40116d0 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.078+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc1a5b90 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.079+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecfc1030b0 con 0x7fecfc109050 2024-09-10T23:29:19.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.082+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fecf400f040 con 0x7fecfc109050 2024-09-10T23:29:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.083+0000 7fecf8c00640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 0x7fece00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.083+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7fecf40e65a0 con 0x7fecfc109050 2024-09-10T23:29:19.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.083+0000 7fecfb400640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 0x7fece00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.084+0000 7fecfb400640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 0x7fece00b7910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fecec005d00 tx=0x7fecec00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:19.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.085+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fecf4016030 con 0x7fecfc109050 2024-09-10T23:29:19.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:18 smithi145 ceph-mon[34397]: from='client.16650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:18 smithi145 ceph-mon[34397]: from='client.16656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:18 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/202184259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:29:19.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:18 smithi092 ceph-mon[34223]: from='client.16650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:18 smithi092 ceph-mon[34223]: from='client.16656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:18 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/202184259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:29:19.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.285+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fecfc02e5c0 con 0x7fecfc109050 2024-09-10T23:29:19.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.286+0000 7fecf8c00640 1 -- 172.21.15.67:0/4098679282 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) ==== 104+0+11 (secure 0 0 0) 0x7fecf40acd40 con 0x7fecfc109050 2024-09-10T23:29:19.287 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-09-10T23:29:19.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 msgr2=0x7fece00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 0x7fece00b7910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fecec005d00 tx=0x7fecec00b410 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 msgr2=0x7fecfc19ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fecf400eaa0 tx=0x7fecf400ef60 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 shutdown_connections 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fece00b5450 0x7fece00b7910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecfc109050 0x7fecfc19ea10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecfc1086d0 0x7fecfc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 --2- 172.21.15.67:0/4098679282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecfc102cd0 0x7fecfc199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 >> 172.21.15.67:0/4098679282 conn(0x7fecfc0fe710 msgr2=0x7fecfc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.288+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 shutdown_connections 2024-09-10T23:29:19.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.289+0000 7fed02200640 1 -- 172.21.15.67:0/4098679282 wait complete. 2024-09-10T23:29:19.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:18 smithi067 ceph-mon[30939]: from='client.16650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:18 smithi067 ceph-mon[30939]: from='client.16656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:29:19.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:18 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/202184259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:29:19.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 -- 172.21.15.67:0/136582640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10a8e0 msgr2=0x7f017c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 --2- 172.21.15.67:0/136582640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10a8e0 0x7f017c10ad20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f016402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 -- 172.21.15.67:0/136582640 shutdown_connections 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 --2- 172.21.15.67:0/136582640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10b260 0x7f017c111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 --2- 172.21.15.67:0/136582640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10a8e0 0x7f017c10ad20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 --2- 172.21.15.67:0/136582640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 -- 172.21.15.67:0/136582640 >> 172.21.15.67:0/136582640 conn(0x7f017c100920 msgr2=0x7f017c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:19.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 -- 172.21.15.67:0/136582640 shutdown_connections 2024-09-10T23:29:19.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.466+0000 7f0182600640 1 -- 172.21.15.67:0/136582640 wait complete. 2024-09-10T23:29:19.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 Processor -- start 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 -- start start 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10a8e0 0x7f017c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f017c114d90 con 0x7f017c10b260 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f017c114c10 con 0x7f017c10a8e0 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0182600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f017c114f10 con 0x7f017c104ee0 2024-09-10T23:29:19.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f017be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f017be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58822/0 (socket says 172.21.15.67:58822) 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f017be00640 1 -- 172.21.15.67:0/132908151 learned_addr learned my addr 172.21.15.67:0/132908151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.467+0000 7f0172a00640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10a8e0 0x7f017c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017b400640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 -- 172.21.15.67:0/132908151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 msgr2=0x7f017c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 -- 172.21.15.67:0/132908151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10a8e0 msgr2=0x7f017c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10a8e0 0x7f017c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 -- 172.21.15.67:0/132908151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c1a13f0 con 0x7f017c10b260 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017b400640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f017be00640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f017400eaa0 tx=0x7f017400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017400cdb0 con 0x7f017c10b260 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0174005020 con 0x7f017c10b260 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740116d0 con 0x7f017c10b260 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c1a16e0 con 0x7f017c10b260 2024-09-10T23:29:19.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.468+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a7da0 con 0x7f017c10b260 2024-09-10T23:29:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.469+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c1052c0 con 0x7f017c10b260 2024-09-10T23:29:19.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.472+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f017400f040 con 0x7f017c10b260 2024-09-10T23:29:19.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.473+0000 7f0178c00640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 0x7f01600b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.473+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7f01740e6210 con 0x7f017c10b260 2024-09-10T23:29:19.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.473+0000 7f017b400640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 0x7f01600b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.474+0000 7f017b400640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 0x7f01600b7990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f016c005d00 tx=0x7f016c005ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:19.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.477+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0174016030 con 0x7f017c10b260 2024-09-10T23:29:19.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.677+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f017c02e5c0 con 0x7f017c10b260 2024-09-10T23:29:19.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.678+0000 7f0178c00640 1 -- 172.21.15.67:0/132908151 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) ==== 100+0+11 (secure 0 0 0) 0x7f01740ac950 con 0x7f017c10b260 2024-09-10T23:29:19.679 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-09-10T23:29:19.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 msgr2=0x7f01600b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 0x7f01600b7990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f016c005d00 tx=0x7f016c005ed0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 msgr2=0x7f017c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f017400eaa0 tx=0x7f017400ef60 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 shutdown_connections 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f01600b54d0 0x7f01600b7990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f017c10b260 0x7f017c1a0c20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f017c10a8e0 0x7f017c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 --2- 172.21.15.67:0/132908151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f017c104ee0 0x7f017c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 >> 172.21.15.67:0/132908151 conn(0x7f017c100920 msgr2=0x7f017c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 shutdown_connections 2024-09-10T23:29:19.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.680+0000 7f0182600640 1 -- 172.21.15.67:0/132908151 wait complete. 2024-09-10T23:29:19.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T23:29:19.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 -- 172.21.15.67:0/2391368969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff090104ee0 msgr2=0x7ff0901052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 --2- 172.21.15.67:0/2391368969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff090104ee0 0x7ff0901052c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002a410 comp rx=0 tx=0).stop 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 -- 172.21.15.67:0/2391368969 shutdown_connections 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 --2- 172.21.15.67:0/2391368969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 0x7ff090111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 --2- 172.21.15.67:0/2391368969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff09010a8e0 0x7ff09010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 --2- 172.21.15.67:0/2391368969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff090104ee0 0x7ff0901052c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 -- 172.21.15.67:0/2391368969 >> 172.21.15.67:0/2391368969 conn(0x7ff090100920 msgr2=0x7ff090102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 -- 172.21.15.67:0/2391368969 shutdown_connections 2024-09-10T23:29:19.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.859+0000 7ff096400640 1 -- 172.21.15.67:0/2391368969 wait complete. 2024-09-10T23:29:19.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.860+0000 7ff096400640 1 Processor -- start 2024-09-10T23:29:19.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.860+0000 7ff096400640 1 -- start start 2024-09-10T23:29:19.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.860+0000 7ff096400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090104ee0 0x7ff09019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.860+0000 7ff096400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff096400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 0x7ff0901a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff096400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114da0 con 0x7ff09010a8e0 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff096400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114c20 con 0x7ff090104ee0 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff096400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114f20 con 0x7ff09010b260 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58830/0 (socket says 172.21.15.67:58830) 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 -- 172.21.15.67:0/89407174 learned_addr learned my addr 172.21.15.67:0/89407174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08f400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090104ee0 0x7ff09019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08fe00640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 0x7ff0901a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 -- 172.21.15.67:0/89407174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 msgr2=0x7ff0901a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 0x7ff0901a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 -- 172.21.15.67:0/89407174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090104ee0 msgr2=0x7ff09019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090104ee0 0x7ff09019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 -- 172.21.15.67:0/89407174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a1460 con 0x7ff09010a8e0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.861+0000 7ff08ea00640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d950 tx=0x7ff07c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.862+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c017070 con 0x7ff09010a8e0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.862+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff07c009c30 con 0x7ff09010a8e0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.862+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00f620 con 0x7ff09010a8e0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.862+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a1750 con 0x7ff09010a8e0 2024-09-10T23:29:19.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.862+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a7ff0 con 0x7ff09010a8e0 2024-09-10T23:29:19.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.863+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0901052c0 con 0x7ff09010a8e0 2024-09-10T23:29:19.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.866+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff07c010460 con 0x7ff09010a8e0 2024-09-10T23:29:19.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.867+0000 7ff087e00640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 0x7ff0740b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:19.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.867+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7ff07c0e5fe0 con 0x7ff09010a8e0 2024-09-10T23:29:19.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.868+0000 7ff08f400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 0x7ff0740b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:19.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.868+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff07c0ac7a0 con 0x7ff09010a8e0 2024-09-10T23:29:19.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:19.868+0000 7ff08f400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 0x7ff0740b7990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff080005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:20.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:19 smithi067 ceph-mon[30939]: pgmap v1347: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 281 B/s wr, 8 op/s 2024-09-10T23:29:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/994722224' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:19 smithi067 ceph-mon[30939]: osdmap e137: 12 total, 12 up, 12 in 2024-09-10T23:29:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/4098679282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:19 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/132908151' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.063+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff09010d0a0 con 0x7ff09010a8e0 2024-09-10T23:29:20.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.063+0000 7ff087e00640 1 -- 172.21.15.67:0/89407174 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v137) ==== 103+0+11 (secure 0 0 0) 0x7ff07c009de0 con 0x7ff09010a8e0 2024-09-10T23:29:20.064 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.065+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 msgr2=0x7ff0740b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.065+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 0x7ff0740b7990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff080005e80 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 msgr2=0x7ff09019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d950 tx=0x7ff07c00de10 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 shutdown_connections 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7ff0740b54d0 0x7ff0740b7990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09010b260 0x7ff0901a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff09010a8e0 0x7ff09019b900 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 --2- 172.21.15.67:0/89407174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090104ee0 0x7ff09019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 >> 172.21.15.67:0/89407174 conn(0x7ff090100920 msgr2=0x7ff090101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:20.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 shutdown_connections 2024-09-10T23:29:20.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.066+0000 7ff096400640 1 -- 172.21.15.67:0/89407174 wait complete. 2024-09-10T23:29:20.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T23:29:20.098 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:19 smithi145 ceph-mon[34397]: pgmap v1347: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 281 B/s wr, 8 op/s 2024-09-10T23:29:20.098 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/994722224' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.098 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:19 smithi145 ceph-mon[34397]: osdmap e137: 12 total, 12 up, 12 in 2024-09-10T23:29:20.098 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/4098679282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.098 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:19 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/132908151' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 -- 172.21.15.67:0/3922212295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac10a900 msgr2=0x7fe5ac10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 --2- 172.21.15.67:0/3922212295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac10a900 0x7fe5ac10ad40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 -- 172.21.15.67:0/3922212295 shutdown_connections 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 --2- 172.21.15.67:0/3922212295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10b280 0x7fe5ac111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 --2- 172.21.15.67:0/3922212295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac10a900 0x7fe5ac10ad40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 --2- 172.21.15.67:0/3922212295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac104f00 0x7fe5ac1052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 -- 172.21.15.67:0/3922212295 >> 172.21.15.67:0/3922212295 conn(0x7fe5ac100920 msgr2=0x7fe5ac102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 -- 172.21.15.67:0/3922212295 shutdown_connections 2024-09-10T23:29:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.242+0000 7fe5b4200640 1 -- 172.21.15.67:0/3922212295 wait complete. 2024-09-10T23:29:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.243+0000 7fe5b4200640 1 Processor -- start 2024-09-10T23:29:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.243+0000 7fe5b4200640 1 -- start start 2024-09-10T23:29:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10a900 0x7fe5ac19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac10b280 0x7fe5ac1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac114d80 con 0x7fe5ac104f00 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac114c00 con 0x7fe5ac10a900 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b4200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac114f00 con 0x7fe5ac10b280 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58842/0 (socket says 172.21.15.67:58842) 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 -- 172.21.15.67:0/1318511468 learned_addr learned my addr 172.21.15.67:0/1318511468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b0e00640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10a900 0x7fe5ac19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b2200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac10b280 0x7fe5ac1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 -- 172.21.15.67:0/1318511468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac10b280 msgr2=0x7fe5ac1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac10b280 0x7fe5ac1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 -- 172.21.15.67:0/1318511468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10a900 msgr2=0x7fe5ac19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.244+0000 7fe5b1800640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10a900 0x7fe5ac19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5b1800640 1 -- 172.21.15.67:0/1318511468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac1a13e0 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5b1800640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d770 tx=0x7fe5a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0017070 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac1a16d0 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a00054b0 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0020060 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.245+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac1a7d90 con 0x7fe5ac104f00 2024-09-10T23:29:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.246+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac1052e0 con 0x7fe5ac104f00 2024-09-10T23:29:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.248+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe5a0004c00 con 0x7fe5ac104f00 2024-09-10T23:29:20.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.250+0000 7fe5aa000640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 0x7fe5940b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.250+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7fe5a00e5f10 con 0x7fe5ac104f00 2024-09-10T23:29:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.250+0000 7fe5b0e00640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 0x7fe5940b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:20.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.251+0000 7fe5b0e00640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 0x7fe5940b7910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5980057c0 tx=0x7fe5980058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.253+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe5a00ac000 con 0x7fe5ac104f00 2024-09-10T23:29:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:19 smithi092 ceph-mon[34223]: pgmap v1347: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 281 B/s wr, 8 op/s 2024-09-10T23:29:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/994722224' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:19 smithi092 ceph-mon[34223]: osdmap e137: 12 total, 12 up, 12 in 2024-09-10T23:29:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/4098679282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:19 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/132908151' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:29:20.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.499+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe5ac19c460 con 0x7fe5ac104f00 2024-09-10T23:29:20.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.510+0000 7fe5aa000640 1 -- 172.21.15.67:0/1318511468 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+828473 (secure 0 0 0) 0x7fe5a00b0a30 con 0x7fe5ac104f00 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 24, 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: "active_gid": 14223, 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: "active_name": "x", 2024-09-10T23:29:20.525 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addrs": { 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6800", 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1232465191 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6801", 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1232465191 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:20.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addr": "172.21.15.67:6801/1232465191", 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "active_change": "2024-09-10T22:48:31.958262+0000", 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "standbys": [ 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24169, 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "z", 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-10T23:29:20.527 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-10T23:29:20.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-10T23:29:20.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-10T23:29:20.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.544 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.549 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:29:20.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-10T23:29:20.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:29:20.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-10T23:29:20.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-10T23:29:20.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.557 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.558 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.562 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-10T23:29:20.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:20.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-10T23:29:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:20.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-10T23:29:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:29:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.580 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:29:20.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-10T23:29:20.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:29:20.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:29:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:29:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-10T23:29:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-10T23:29:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:29:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-10T23:29:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-10T23:29:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:29:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-10T23:29:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.612 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:29:20.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-10T23:29:20.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.620 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.632 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.634 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-10T23:29:20.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:29:20.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-10T23:29:20.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-10T23:29:20.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.647 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.651 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.652 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.655 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.656 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:29:20.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.659 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.661 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:29:20.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.668 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:29:20.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:29:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.671 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:20.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:29:20.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:29:20.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.686 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:29:20.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:29:20.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:29:20.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:29:20.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.713 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.714 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:29:20.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-10T23:29:20.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.733 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-10T23:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-10T23:29:20.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.747 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-10T23:29:20.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-10T23:29:20.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:29:20.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-10T23:29:20.758 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.762 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:20.764 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-10T23:29:20.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-10T23:29:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-10T23:29:20.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.778 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-10T23:29:20.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-10T23:29:20.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-10T23:29:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-10T23:29:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-10T23:29:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-10T23:29:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-10T23:29:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:29:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-10T23:29:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-10T23:29:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:29:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.928 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.934 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.941 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.942 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:29:20.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:29:20.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.945 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.947 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.954 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:29:20.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.958 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.959 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.961 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.978 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.980 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.981 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.983 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-10T23:29:20.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-10T23:29:20.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.989 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.990 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:20.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:20.996 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:20.997 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:20.998 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:20.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.004 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.006 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.009 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-10T23:29:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.011 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.012 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.013 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.014 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.016 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.017 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.020 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-10T23:29:21.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:29:21.024 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.025 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.026 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-10T23:29:21.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.030 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.033 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.035 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.037 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.040 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-10T23:29:21.043 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.044 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.047 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.048 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.049 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.051 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.059 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.064 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.071 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.072 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.074 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.076 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.077 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.078 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-10T23:29:21.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 34163, 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "y", 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-10T23:29:21.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.083 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.085 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.088 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-10T23:29:21.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.092 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.094 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.099 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:29:21.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-10T23:29:21.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.104 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-10T23:29:21.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-10T23:29:21.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.109 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.112 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-10T23:29:21.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:29:21.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:29:21.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-10T23:29:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:29:21.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-10T23:29:21.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:29:21.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.136 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:29:21.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.139 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-10T23:29:21.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-10T23:29:21.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-10T23:29:21.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-10T23:29:21.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.152 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.154 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.155 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-10T23:29:21.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-10T23:29:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.173 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.175 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.176 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.177 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.181 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.182 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.184 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-10T23:29:21.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:29:21.189 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-10T23:29:21.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-10T23:29:21.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.193 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.194 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.197 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.198 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.204 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.205 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.206 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.209 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.210 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.214 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.216 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.217 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.222 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.224 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.232 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.233 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.234 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:29:21.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:29:21.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.237 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.238 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.241 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.244 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.247 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:29:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:29:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-10T23:29:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.259 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.265 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.266 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.268 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.270 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.272 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.273 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.275 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.277 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.282 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-10T23:29:21.286 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.289 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.290 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.291 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.293 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.295 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.296 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-10T23:29:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:29:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-10T23:29:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-10T23:29:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-10T23:29:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.315 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:29:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.320 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.324 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.326 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:29:21.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.334 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.338 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.339 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.343 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.347 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.348 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.351 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-10T23:29:21.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.353 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.354 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.355 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.358 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-10T23:29:21.362 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.365 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-10T23:29:21.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-10T23:29:21.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-10T23:29:21.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.374 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.375 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.376 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.379 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.384 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.387 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.390 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.391 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.392 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.393 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.395 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.396 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:21.399 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.401 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.403 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.404 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.405 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.420 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.424 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.425 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.431 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-10T23:29:21.434 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:29:21.435 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.437 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.440 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.445 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.446 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:29:21.447 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:29:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.456 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:21.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.464 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.466 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.470 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.474 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.476 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.482 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-10T23:29:21.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.491 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.496 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.499 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.501 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.503 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.506 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.507 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.510 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.511 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-10T23:29:21.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.514 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.515 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.517 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.520 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.527 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.532 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.536 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-10T23:29:21.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-10T23:29:21.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.539 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.540 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.551 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.553 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.565 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.568 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-10T23:29:21.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-10T23:29:21.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-10T23:29:21.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.588 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.593 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.606 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-10T23:29:21.612 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:29:21.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:29:21.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.616 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.617 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-10T23:29:21.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.623 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.624 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.627 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.631 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard", 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "restful", 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "snap_schedule" 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-09-10T23:29:21.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.635 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.638 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-09-10T23:29:21.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-09-10T23:29:21.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-09-10T23:29:21.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:21.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-09-10T23:29:21.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-09-10T23:29:21.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:29:21.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:29:21.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.659 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.661 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.664 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-09-10T23:29:21.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.668 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.675 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:29:21.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:29:21.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-09-10T23:29:21.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-09-10T23:29:21.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.687 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.688 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:29:21.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.694 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.697 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.698 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.699 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.700 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.701 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-09-10T23:29:21.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-09-10T23:29:21.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:29:21.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba_metrics": { 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:29:21.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.721 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.723 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-09-10T23:29:21.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-09-10T23:29:21.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.735 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:29:21.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:29:21.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-09-10T23:29:21.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.742 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.747 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.755 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-09-10T23:29:21.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.759 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.761 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.764 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.765 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:29:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:29:21.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.768 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:29:21.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.772 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.773 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.782 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:29:21.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.796 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:29:21.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.800 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.801 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.803 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:29:21.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.805 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.807 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.815 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-09-10T23:29:21.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-09-10T23:29:21.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.825 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.828 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:21.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-09-10T23:29:21.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:21.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-09-10T23:29:21.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.844 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.845 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-09-10T23:29:21.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-09-10T23:29:21.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:29:21.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.857 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.858 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.861 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-09-10T23:29:21.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.864 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-09-10T23:29:21.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-09-10T23:29:21.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.873 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.876 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.877 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.878 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-09-10T23:29:21.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-09-10T23:29:21.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.883 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:29:21.885 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.891 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.893 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.896 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.898 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.902 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.904 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.905 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.908 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.910 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.911 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-09-10T23:29:21.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.914 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.916 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:29:21.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-09-10T23:29:21.922 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-09-10T23:29:21.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.924 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.925 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.928 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.929 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.930 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.932 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.934 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.937 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.941 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.942 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.943 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.945 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.946 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.949 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.953 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.954 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.956 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.959 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.961 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.963 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.971 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.976 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.978 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.979 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-09-10T23:29:21.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.984 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:29:21.985 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:21.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.988 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.989 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:21.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-09-10T23:29:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.993 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:21.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:21.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:21.998 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:21.999 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.000 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.004 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.007 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-09-10T23:29:22.011 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.012 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.013 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.014 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.015 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.016 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.018 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.022 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.024 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.025 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.026 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.028 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.031 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.033 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max_requests": { 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_requests", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.035 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.036 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.038 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.039 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.040 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.042 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.044 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.061 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-09-10T23:29:22.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.066 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.070 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.071 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.072 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.074 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.076 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.077 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.079 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.084 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:29:22.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:29:22.087 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.088 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.090 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.091 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.094 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.095 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.096 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.098 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.099 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.100 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.101 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.102 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.104 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.106 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.110 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.113 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.116 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.118 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.129 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.134 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.135 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.138 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-09-10T23:29:22.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-09-10T23:29:22.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-09-10T23:29:22.144 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.145 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.146 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.152 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.158 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.162 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-09-10T23:29:22.164 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-09-10T23:29:22.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-09-10T23:29:22.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-09-10T23:29:22.171 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-09-10T23:29:22.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.175 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-09-10T23:29:22.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "services": { 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard": "https://172.21.15.67:8443/" 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "always_on_modules": { 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus": [ 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-10T23:29:22.178 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific": [ 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-10T23:29:22.179 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy": [ 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.180 INFO:teuthology.orchestra.run.smithi067.stdout: "reef": [ 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "squid": [ 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-09-10T23:29:22.181 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "force_disabled_modules": {}, 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "last_failure_osd_epoch": 5, 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "active_clients": [ 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2947326901 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "libcephsqlite", 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3158436818 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 910979027 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1162290890 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2131841360 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-09-10T23:29:22.187 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 633327851 2024-09-10T23:29:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:22.188 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-10T23:29:22.188 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 msgr2=0x7fe5940b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 0x7fe5940b7910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5980057c0 tx=0x7fe5980058d0 comp rx=0 tx=0).stop 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 msgr2=0x7fe5ac19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d770 tx=0x7fe5a000dc30 comp rx=0 tx=0).stop 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 shutdown_connections 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe5940b5450 0x7fe5940b7910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5ac10b280 0x7fe5ac1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5ac10a900 0x7fe5ac19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 --2- 172.21.15.67:0/1318511468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5ac104f00 0x7fe5ac19b340 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 >> 172.21.15.67:0/1318511468 conn(0x7fe5ac100920 msgr2=0x7fe5ac102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 shutdown_connections 2024-09-10T23:29:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:20.518+0000 7fe5b4200640 1 -- 172.21.15.67:0/1318511468 wait complete. 2024-09-10T23:29:22.198 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T23:29:22.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T23:29:22.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/89407174' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T23:29:22.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:20 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1318511468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:29:22.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/89407174' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T23:29:22.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:20 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1318511468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:29:22.200 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/89407174' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T23:29:22.200 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:20 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1318511468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:29:22.200 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:21 smithi145 ceph-mon[34397]: pgmap v1349: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 204 B/s wr, 8 op/s 2024-09-10T23:29:22.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:21 smithi067 ceph-mon[30939]: pgmap v1349: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 204 B/s wr, 8 op/s 2024-09-10T23:29:22.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:21 smithi092 ceph-mon[34223]: pgmap v1349: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 204 B/s wr, 8 op/s 2024-09-10T23:29:22.589 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:23.214 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:23.214 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 538, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 531, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 925216, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 328256, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.126039866, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.072407663, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.057713711, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 531, 2024-09-10T23:29:23.215 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.053609734, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000100959 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 925216, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 328256 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:23.216 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 3, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 405, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 480, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 312560, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.052766768, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.043556685, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.004110652, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 403, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.050074509, 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000124254 2024-09-10T23:29:23.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 312560 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:29:23.218 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.000764417, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000625240, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:23.219 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-09-10T23:29:23.220 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:23.221 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:23.222 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148704, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 29, 2024-09-10T23:29:23.223 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-09-10T23:29:23.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-09-10T23:29:23.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 537, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 846766, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 537, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 537, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 846766, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:23.226 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:23.227 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:23.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:23.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:23.227 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:23.950 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:23 smithi067 ceph-mon[30939]: pgmap v1350: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-10T23:29:23.991 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-10T23:29:23.992 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T23:29:23.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T23:29:24.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:23 smithi145 ceph-mon[34397]: pgmap v1350: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:24.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:23 smithi092 ceph-mon[34223]: pgmap v1350: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:24.968 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 3, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.003336805, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000540602, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:24.969 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 548, 2024-09-10T23:29:24.970 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 541, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1082704, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 333056, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.122113400, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.065623551, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.065281585, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 540, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.066611207, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000123354 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.971 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1082704, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 333056 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 3, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 411, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 480, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 315728, 2024-09-10T23:29:24.972 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.055553675, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.047045683, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.003405123, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 409, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.040997591, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000100238 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:24.973 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 315728 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:24.974 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:24.976 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148656, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 27, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:24.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-09-10T23:29:24.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 547, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 1002861, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 547, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 547, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 1002861, 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-09-10T23:29:24.979 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:24.980 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:24.980 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:25.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:25 smithi067 ceph-mon[30939]: pgmap v1351: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:26.017 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T23:29:26.017 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-10T23:29:26.018 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-10T23:29:26.018 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T23:29:26.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T23:29:26.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:25 smithi145 ceph-mon[34397]: pgmap v1351: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:26.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:25 smithi092 ceph-mon[34223]: pgmap v1351: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-10T23:29:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:27.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:29:27.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:26 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:29:27.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:26 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:29:27.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.207 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:26 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:27.581 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:27.581 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 540, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 533, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 925760, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 329536, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.117010007, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.066026281, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.055669728, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 533, 2024-09-10T23:29:27.582 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.054480876, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000102215 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 925760, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 329536 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 3, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 407, 2024-09-10T23:29:27.583 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 480, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 312912, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.055865288, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.047330293, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.003457563, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 405, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.040720484, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000100544 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:27.584 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 312912 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:27.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.000641039, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000494164, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:27.586 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:27.587 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:27.588 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:27.589 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148784, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 31, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-09-10T23:29:27.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-09-10T23:29:27.591 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 539, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 847026, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 539, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-09-10T23:29:27.592 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 539, 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 847026, 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:27.593 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:28.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:27 smithi145 ceph-mon[34397]: pgmap v1352: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T23:29:28.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:27 smithi092 ceph-mon[34223]: pgmap v1352: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T23:29:28.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:27 smithi067 ceph-mon[30939]: pgmap v1352: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-10T23:29:29.148 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-10T23:29:29.149 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-10T23:29:29.149 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-10T23:29:29.149 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T23:29:29.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T23:29:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:29.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:29 smithi067 ceph-mon[30939]: pgmap v1353: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-10T23:29:29.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.709 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:29 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:29 smithi092 ceph-mon[34223]: pgmap v1353: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-10T23:29:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.771 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:29 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:29 smithi145 ceph-mon[34397]: pgmap v1353: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-10T23:29:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:29.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:29 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:30.317 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-10T23:29:30.317 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:30.317 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 543, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 538, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 924720, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 333568, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.120594145, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.071527170, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.054215342, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 537, 2024-09-10T23:29:30.318 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.055211945, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000102815 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 924720, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 333568 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 7, 2024-09-10T23:29:30.319 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 414, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 3424, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 314240, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.061644247, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.050171664, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.006449056, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 413, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.048537913, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000117525 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 314240 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-09-10T23:29:30.321 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.000619144, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000468469, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:30.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:30.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:30.324 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:30.325 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148704, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 29, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:30.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-09-10T23:29:30.327 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 546, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 847936, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 546, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 546, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 847936, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-09-10T23:29:30.328 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-09-10T23:29:30.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:30.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-10T23:29:30.329 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-10T23:29:31.041 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-10T23:29:31.041 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T23:29:31.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: pgmap v1354: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:29:31.347 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:31 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: pgmap v1354: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:29:31.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:31 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:29:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: pgmap v1354: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-10T23:29:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:29:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:29:31.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:31 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 13620, 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 13308, 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 5738253, 2024-09-10T23:29:32.429 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 12271869, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 19, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 5.038757995, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.801437869, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.898136910, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 3.391920526, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13304, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.726095897, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000129742 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:32.430 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 21631, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004831612, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000223 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2111984, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 714304 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 6918, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 11007, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2034662, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 9758811, 2024-09-10T23:29:32.431 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 101, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.450732086, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.900106140, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.527812282, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.054768102, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11009, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.393333498, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000217397 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13213, 2024-09-10T23:29:32.432 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006264756, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000474 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 879648 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 7006, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 8088, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2325539, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 14697608, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 12, 2024-09-10T23:29:32.433 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.354041113, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.803742361, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.489660655, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.109914793, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8086, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.605616476, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000198567 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9528, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003823169, 2024-09-10T23:29:32.434 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000401 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2423, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.212107134, 2024-09-10T23:29:32.435 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000912962 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2763, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.209706356, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000075898 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 4630, 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 4254, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4254, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 74.872520096, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017600498 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 376, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 1, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 12, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-10T23:29:32.437 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 880, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 244, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 636, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 879, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 508, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 3, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 3, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 6991, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 6241, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 375, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 8, 2024-09-10T23:29:32.438 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 1, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 51, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 1, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 8, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 516, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 8, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 353, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-09-10T23:29:32.439 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 5, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 1564, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 1874, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 6, 2024-09-10T23:29:32.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 3463, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 1083, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 33, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 2561 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 287, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 11, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 656, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 81, 2024-09-10T23:29:32.441 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 840, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 698, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 61, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 22, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 4093, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:32.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 8, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_write_failed": 0 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 5014, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-10T23:29:32.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 5014, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 8, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 8, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 23857613, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-10T23:29:32.444 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1211, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 109.012615060, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.090018674 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 327, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1404, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 1077, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 101, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 216, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 115, 2024-09-10T23:29:32.445 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 880, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 2133, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 1253, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 3, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 982, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 979, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 65520, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 281931776 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 6066, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 20, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 4630, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 56, 2024-09-10T23:29:32.446 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 67, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 643, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.660630315, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001027418 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1439, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.409466553, 2024-09-10T23:29:32.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000284549 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 61, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.132581466, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002173466 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 527, 2024-09-10T23:29:32.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 43.748224422, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.083013708 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-10T23:29:32.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.450 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 30, 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006721792, 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000224059 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 105, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.107142151, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001020401 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.452 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 30, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 12.592665309, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.419755510 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 207, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.890448993, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.018794439 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 238, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.727498559, 2024-09-10T23:29:32.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003056716 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 108, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 5.123716833, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.047441822 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:32.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 97, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.633179009, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.037455453 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 24, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.641265044, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.026719376 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.455 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004069713, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000813942 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.456 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001238112, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000309528 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.025287495, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003160936 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.009851490, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000757806 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 715, 2024-09-10T23:29:32.458 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.158532840, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004417528 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-09-10T23:29:32.459 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 4, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 761, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 380, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1786, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:32.460 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:32.461 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:32.462 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 976615, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 270, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:32.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 198, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 1687243, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 28789, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.464 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 5012, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 19831407, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 5012, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5012, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 360.926711122, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.072012512 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-10T23:29:32.465 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5012, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 5250 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 5012, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 53, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 4959, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 11, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 52, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1263, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 128, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-10T23:29:32.466 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 3361, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 22, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-10T23:29:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 6, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-10T23:29:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 137, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 58, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 16, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 4, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 316, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 89 2024-09-10T23:29:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 52, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 10, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 2845, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 698, 2024-09-10T23:29:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 27540, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 7475728, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 27540, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 27540, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 7475728, 2024-09-10T23:29:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 5012, 2024-09-10T23:29:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 19918617, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5012, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 19918617, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 5012, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 5012, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5012, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 5012, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 698, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 87226, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 698, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 52, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 87226, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55c81a5f8f00": { 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5014, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 19663309, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5014, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1211, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 19663309, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "trackedop": { 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 3791 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:32.477 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-10T23:29:33.280 INFO:tasks.check_counter:Daemon mds.b mds.exported=8 2024-09-10T23:29:33.280 INFO:tasks.check_counter:Daemon mds.b mds.imported=8 2024-09-10T23:29:33.280 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=287 2024-09-10T23:29:33.280 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-10T23:29:33.280 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-10T23:29:33.281 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=30 2024-09-10T23:29:33.281 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=24 2024-09-10T23:29:33.281 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-09-10T23:29:33.281 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T23:29:33.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T23:29:33.652 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:29:34.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:33 smithi145 ceph-mon[34397]: pgmap v1355: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T23:29:34.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:33 smithi092 ceph-mon[34223]: pgmap v1355: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T23:29:34.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:33 smithi067 ceph-mon[30939]: pgmap v1355: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.003462846, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000881931, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 547, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 540, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 927664, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 334016, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.102332258, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.047827938, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.062251715, 2024-09-10T23:29:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 540, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.052192070, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000096651 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 927664, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 334016 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 412, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 480, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 245024, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.038276314, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.029448576, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.002470163, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 410, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.041797389, 2024-09-10T23:29:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000101944 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 245024 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 150766, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 31, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-09-10T23:29:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 546, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 847936, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 546, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 546, 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 847936, 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:34.688 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-10T23:29:35.577 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-10T23:29:35.578 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-10T23:29:35.578 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T23:29:35.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T23:29:35.961 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:29:36.099 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:35 smithi092 ceph-mon[34223]: pgmap v1356: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:35 smithi145 ceph-mon[34397]: pgmap v1356: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:36.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:35 smithi067 ceph-mon[30939]: pgmap v1356: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:36.680 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-10T23:29:36.680 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:36.680 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 8083, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 12129, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2385697, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 19875653, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 15, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.294683761, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.694910658, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.645822376, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.061812879, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 12127, 2024-09-10T23:29:36.681 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.100990375, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000090788 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 15349, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010085004, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000657 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 736 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.682 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 5994, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 5317, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2583630, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 2759221, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.965075787, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.581366424, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.461462926, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.027449041, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5317, 2024-09-10T23:29:36.683 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.483684774, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000090969 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9872, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002677031, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000271 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 858704 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:36.684 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2153, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2130, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1515428, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5148701, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.381421025, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.172251224, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.120471528, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.082782455, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2128, 2024-09-10T23:29:36.685 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.324802801, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000152632 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000200, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000100 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1090000, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 347920 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-10T23:29:36.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2961, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.916204747, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000309424 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1762, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.083297741, 2024-09-10T23:29:36.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000047274 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 1078, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 883, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 883, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 43.047470030, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.048751381 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 195, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 4, 2024-09-10T23:29:36.688 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 20, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 1, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 772, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 112, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 647, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 13, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 610, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 555, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 3, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 3, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 4, 2024-09-10T23:29:36.689 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 2042, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 1295, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 159, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 155, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 3, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 9, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 350, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 23, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 778, 2024-09-10T23:29:36.690 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 287, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 9352036406, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:36.691 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 9, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 636, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 1165, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 1906, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 1203, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 5, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 216 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 85, 2024-09-10T23:29:36.692 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 464, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 309, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 311, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 99, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 2, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 51, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 49, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:36.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 22, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 3156, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 10, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 1, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:36.694 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_write_failed": 0 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 2049, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 2049, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 6, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-10T23:29:36.695 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 6, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 24391683, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1329, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 94.067748026, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.070780848 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.696 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 649, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1069, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 420, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 43, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 214, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 171, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 772, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1376, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 604, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 3, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 294, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 291, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 60868, 2024-09-10T23:29:36.697 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 269348864 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 1182, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 78, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 1078, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 33, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 184, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 100, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.138177965, 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001381779 2024-09-10T23:29:36.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003513197, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000439149 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.699 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 533, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 42.515204503, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.079765862 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.700 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.701 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.702 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:36.702 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.702 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.702 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-10T23:29:36.703 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 182, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.280748044, 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001542571 2024-09-10T23:29:36.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006433008, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000714778 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.016975581, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003395116 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.705 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:36.706 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.706 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.706 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.706 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:36.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.708 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.017626434, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008813217 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 44, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.068791298, 2024-09-10T23:29:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001563438 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-10T23:29:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 134, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 134, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1809, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 304711, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 103, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 198, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 2023197, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 8162, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 3598, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 20381855, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 28, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 3598, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3598, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 315.084014864, 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.087571988 2024-09-10T23:29:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3598, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3993 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 3598, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 11, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 3587, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 7, 2024-09-10T23:29:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1336, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 108, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 1, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 1768, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-10T23:29:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 137, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 52, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-09-10T23:29:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 14, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 2, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 581, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 8, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 154 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 52, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 4, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 1752, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 49, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 16226, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 4988324, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 16226, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 16226, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 4988324, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3598, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 20389284, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3598, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 20389284, 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3598, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 3598, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3598, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 3598, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 49, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 7493, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 49, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 7, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 7493, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x564e14f96dc0": { 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2049, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 20197379, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2049, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1329, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 20197379, 2024-09-10T23:29:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "trackedop": { 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 3117 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:36.732 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-10T23:29:37.847 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-09-10T23:29:37.847 INFO:tasks.check_counter:Daemon mds.h mds.imported=23 2024-09-10T23:29:37.847 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=85 2024-09-10T23:29:37.848 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=464 2024-09-10T23:29:37.848 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-10T23:29:37.848 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-10T23:29:37.848 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T23:29:37.848 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=1 2024-09-10T23:29:37.848 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T23:29:37.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T23:29:38.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:37 smithi145 ceph-mon[34397]: pgmap v1357: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:38.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:37 smithi092 ceph-mon[34223]: pgmap v1357: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:38.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:37 smithi067 ceph-mon[30939]: pgmap v1357: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 3607, 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4281, 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1965210, 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 3941643, 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-09-10T23:29:38.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.645104615, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.325907993, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.262994766, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.069317453, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4280, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.780938816, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000182462 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3338, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000922861, 2024-09-10T23:29:38.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000276 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1084704, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 351152 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 6338, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4476, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2040782, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 2229207, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-10T23:29:38.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.738288141, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.423308880, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.334273122, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.042198873, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4475, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.411419585, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000091937 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8247, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003279936, 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000397 2024-09-10T23:29:38.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 917648 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 9622, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 5988, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3137438, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1826972, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 14, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.168047723, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.685325518, 2024-09-10T23:29:38.995 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.595514880, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.034150917, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5986, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.675612237, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000112865 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13678, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.004605747, 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000336 2024-09-10T23:29:38.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3488, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 18272 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1338, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.455394788, 2024-09-10T23:29:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000340354 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1666, 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.098210004, 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000058949 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:38.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 643, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 585, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 585, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 29.157510505, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.049841898 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 58, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 2, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 27, 2024-09-10T23:29:38.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 2, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 20, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 2, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 8, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 10, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 14, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 529, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 4, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 1398, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 1044, 2024-09-10T23:29:39.000 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 58, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 133, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 2, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 17, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 311, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 3, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 46, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 287, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 9352036406, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 11, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 368, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 503, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 982, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 594, 2024-09-10T23:29:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 2, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 108 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 562, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 345, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 436, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 3, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 43, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 43, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 22, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 2144, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 19, 2024-09-10T23:29:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 2, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_write_failed": 0 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 1490, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 1490, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 3, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-10T23:29:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 3, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 6165681, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 575, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 99.956712045, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.173837760 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 26, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 467, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 441, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 43, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 265, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 222, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 21, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 593, 2024-09-10T23:29:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 572, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 69, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 69, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 57304, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 266199040 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 763, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 8, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 643, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 43, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 70, 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-10T23:29:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 22, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.063499005, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002886318 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.634072541, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.317036270 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 284, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 28.059473768, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.098800963 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 204, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.338046721, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001657091 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 30, 2024-09-10T23:29:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010831960, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000361065 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-10T23:29:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 43, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.051586510, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001199686 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 149, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 74, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1794, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 314455, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 113, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-10T23:29:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 198, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 156837, 2024-09-10T23:29:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 784, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 2535, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 2134206, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 28, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 2535, 2024-09-10T23:29:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2535, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 314.730025845, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.124153856 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2535, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2770 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 2535, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 5, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 2530, 2024-09-10T23:29:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 3, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 578, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 57, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 1678, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-10T23:29:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-10T23:29:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 137, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 52, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 18, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 6, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-10T23:29:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 343, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 4, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 77 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 52, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-10T23:29:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 1, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 1664, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 43, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 19563, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 5345732, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 19563, 2024-09-10T23:29:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 19563, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 5345732, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2535, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2140925, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2535, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2140925, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2535, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2535, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2535, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2535, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 43, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 6751, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 43, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 6751, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x557f665badc0": { 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1490, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1971377, 2024-09-10T23:29:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1490, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 575, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1971377, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "trackedop": { 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 2036 2024-09-10T23:29:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-10T23:29:39.042 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds.exported=17 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds.imported=3 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=562 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T23:29:39.946 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=2 2024-09-10T23:29:39.946 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T23:29:39.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T23:29:40.088 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:39 smithi145 ceph-mon[34397]: pgmap v1358: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:40.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:39 smithi092 ceph-mon[34223]: pgmap v1358: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:40.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:39 smithi067 ceph-mon[30939]: pgmap v1358: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:40.427 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:29:41.124 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-10T23:29:41.124 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:41.124 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 5899, 2024-09-10T23:29:41.124 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 5900, 2024-09-10T23:29:41.124 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 17656590, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 1875571, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.747926491, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.336307815, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.179020211, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.176426464, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5898, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.336407233, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000057037 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.125 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 323, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 730, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 4277592, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 1086454, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-09-10T23:29:41.126 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.095056346, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.044393662, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.027013007, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.014446887, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 729, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.071849677, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000098559 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.127 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 985904 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 812, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 787, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1119046, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 404608, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-10T23:29:41.128 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.154737458, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.085168946, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.068035519, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.007418951, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 786, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.082861124, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000105421 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1071792, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 343552 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 10925, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.312958414, 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000028646 2024-09-10T23:29:41.130 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-09-10T23:29:41.131 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 814, 2024-09-10T23:29:41.132 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 112, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 702, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 655, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:41.133 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:41.134 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-09-10T23:29:41.135 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-09-10T23:29:41.136 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:41.137 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_started": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:41.138 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_write_failed": 0 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 2049, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-10T23:29:41.139 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 6, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 24391683, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 24388605, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 2049 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 640, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 971, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 331, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 26, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 117, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 91, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 814, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 1190, 2024-09-10T23:29:41.141 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 376, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 53000, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 254599168 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-09-10T23:29:41.142 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.143 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.144 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.146 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-10T23:29:41.147 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.148 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.149 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.150 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:41.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.153 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-10T23:29:41.156 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 2, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 1, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 1, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-09-10T23:29:41.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3086, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:41.158 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:41.159 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:41.160 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 199031, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 58, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:41.161 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 198, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-10T23:29:41.162 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 1982608, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 5815, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 6406, 2024-09-10T23:29:41.163 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 6406, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6406, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 33.640266858, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.005251368 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6406, 2024-09-10T23:29:41.164 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6407 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 6406, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 6406, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 3750, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 2654, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-09-10T23:29:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:41.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-10T23:29:41.167 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-10T23:29:41.168 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 137, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 52, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 6, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 9, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 3, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-09-10T23:29:41.169 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-10T23:29:41.170 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 7030, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 22385842, 2024-09-10T23:29:41.171 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 7030, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 7030, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 22385842, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6406, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 20197384, 2024-09-10T23:29:41.173 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6406, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 20197384, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-10T23:29:41.174 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6406, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 6406, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6406, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 6406, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.175 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:41.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x5579a6173040": { 2024-09-10T23:29:41.177 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:41.178 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: "trackedop": { 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:41.179 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-10T23:29:41.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:41 smithi145 ceph-mon[34397]: pgmap v1359: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T23:29:41.955 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-10T23:29:41.955 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T23:29:41.956 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T23:29:42.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:41 smithi092 ceph-mon[34223]: pgmap v1359: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:42.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:41 smithi067 ceph-mon[30939]: pgmap v1359: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:42.327 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2217, 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2217, 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2550014, 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 704246, 2024-09-10T23:29:42.953 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.299543176, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.119207335, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.079218197, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.072399110, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2216, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.139891978, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000063128 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.954 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2405, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2380, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1510950, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 911509, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-10T23:29:42.955 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.378097112, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.165955882, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.123133788, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.080440422, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2379, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.151780507, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000063800 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:42.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1072064, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 344192 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1915, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2325, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 643088, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 1503598, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:42.957 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.360856712, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.168680519, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.076252256, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.080034168, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2323, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.205806319, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000088595 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 895264 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-10T23:29:42.959 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9852, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.322995989, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000032784 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:42.960 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 44, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 44, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 44, 2024-09-10T23:29:42.962 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-10T23:29:42.963 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-10T23:29:42.964 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-10T23:29:42.964 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-09-10T23:29:42.965 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-09-10T23:29:42.965 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-09-10T23:29:42.965 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-09-10T23:29:42.965 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:42.965 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:42.966 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-10T23:29:42.968 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-10T23:29:42.969 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:42.970 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:42.971 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_started": 0, 2024-09-10T23:29:42.971 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:42.971 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_write_failed": 0 2024-09-10T23:29:42.971 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.971 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 1490, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 3, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-09-10T23:29:42.972 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 6165681, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 6163076, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 1490 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-10T23:29:42.973 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 4, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 323, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 319, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 2, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 31, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 29, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 44, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 363, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 319, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 49816, 2024-09-10T23:29:42.974 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 252497920 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-10T23:29:42.975 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-10T23:29:42.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.981 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.982 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.983 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-10T23:29:42.984 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-10T23:29:42.985 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-09-10T23:29:42.986 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 2, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3087, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:42.987 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:42.988 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:42.989 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 191719, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 55, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:42.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 198, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 43928, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 208, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.991 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 5910, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 5910, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5910, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 40.167124717, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.006796467 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-10T23:29:42.992 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5910, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 5911 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 5910, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 5910, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 3695, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 2213, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-09-10T23:29:42.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-10T23:29:42.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-10T23:29:42.995 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 137, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 52, 2024-09-10T23:29:42.996 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 5, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 8, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 3, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-10T23:29:42.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 6533, 2024-09-10T23:29:42.998 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 4081387, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 6533, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6533, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4081387, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:42.999 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 5910, 2024-09-10T23:29:43.000 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 1971382, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 5910, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1971382, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:43.001 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:43.002 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:43.002 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:43.003 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 5910, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 5910, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 5910, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 5910, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:43.004 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:43.005 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x5562d3ded040": { 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:43.006 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "trackedop": { 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:43.007 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T23:29:43.741 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-10T23:29:43.742 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T23:29:43.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T23:29:44.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:43 smithi145 ceph-mon[34397]: pgmap v1360: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:44.129 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:29:44.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:43 smithi092 ceph-mon[34223]: pgmap v1360: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:44.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:43 smithi067 ceph-mon[30939]: pgmap v1360: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:44.778 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-10T23:29:44.778 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:44.778 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2550, 2024-09-10T23:29:44.778 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2552, 2024-09-10T23:29:44.778 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 7881524, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 810910, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 7, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.409634841, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.164209836, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.106583444, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.099879181, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2550, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.210373978, 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000082499 2024-09-10T23:29:44.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2820, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 3228, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 9103084, 2024-09-10T23:29:44.780 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 1791102, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.383164632, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.183050636, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.089324375, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.081780704, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3226, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.225627122, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000069940 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.781 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 894912 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1299, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1271, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 5440469, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 557853, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-10T23:29:44.782 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.229017290, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.123273092, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.086623509, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.019698553, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1271, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.100280450, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000078898 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.783 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1074624, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 343520 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 10081, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.299936426, 2024-09-10T23:29:44.784 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000029752 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-09-10T23:29:44.785 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 1572, 2024-09-10T23:29:44.786 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 1572, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 1572, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 2, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-10T23:29:44.787 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 4, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 234, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:29:44.788 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:29:44.789 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-09-10T23:29:44.790 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:29:44.791 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_started": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_succeeded": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "uninline_write_failed": 0 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-10T23:29:44.792 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 5014, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 8, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-10T23:29:44.793 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 23857613, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 23857477, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 5014 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 320, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 1409, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 1089, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 94, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 190, 2024-09-10T23:29:44.794 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 96, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 1572, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 2137, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 565, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 53084, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 255651840 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-09-10T23:29:44.795 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.796 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-10T23:29:44.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.798 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-10T23:29:44.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-10T23:29:44.800 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.802 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-10T23:29:44.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.804 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-10T23:29:44.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3087, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-10T23:29:44.808 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:44.809 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:44.810 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 200046, 2024-09-10T23:29:44.811 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 58, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9744, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 198, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:44.812 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 2131699, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 27235, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 6042, 2024-09-10T23:29:44.813 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 26, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 4, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 6042, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6042, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 81.745216051, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.013529496 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6042, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6043 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 6042, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 6042, 2024-09-10T23:29:44.814 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 3742, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 2297, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-10T23:29:44.815 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 2, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:29:44.816 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-10T23:29:44.817 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 137, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 52, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 6, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 8, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 2, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-09-10T23:29:44.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-10T23:29:44.819 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 6665, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 21790590, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 6665, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6665, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 21790590, 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:44.820 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:44.821 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6042, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 19663319, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6042, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 19663319, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:44.822 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6042, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 6042, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6042, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 6042, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:29:44.823 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.824 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x5625d8737040": { 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:44.825 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "trackedop": { 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:44.826 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-10T23:29:45.758 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-10T23:29:45.758 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T23:29:45.759 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-10T23:29:45.759 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T23:29:45.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T23:29:46.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:45 smithi092 ceph-mon[34223]: pgmap v1361: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:46.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:45 smithi145 ceph-mon[34397]: pgmap v1361: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:46.133 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:29:46.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:45 smithi067 ceph-mon[30939]: pgmap v1361: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 4, 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 4, 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2944, 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 448, 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-09-10T23:29:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.005639621, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000061376, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.003331460, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.006053545, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002017848 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 549, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 544, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 926352, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 337408, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.123331821, 2024-09-10T23:29:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.074686255, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.054885106, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 543, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.047729136, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000087898 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 926352, 2024-09-10T23:29:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 337408 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 417, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 480, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 314672, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.041987355, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.032815896, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.002122103, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:29:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 415, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.042268661, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000101852 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 314672 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-10T23:29:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:29:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:29:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148721, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 30, 2024-09-10T23:29:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-10T23:29:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 968, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-10T23:29:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 552, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 848716, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 552, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 552, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 848716, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-10T23:29:46.743 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-10T23:29:47.526 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-10T23:29:47.526 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T23:29:47.537 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T23:29:47.537 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T23:29:47.537 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:29:47.607 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T23:29:47.607 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:29:47.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T23:29:48.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:47 smithi067 ceph-mon[30939]: pgmap v1362: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:48.115 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:48.129 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:47 smithi145 ceph-mon[34397]: pgmap v1362: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:48.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:47 smithi092 ceph-mon[34223]: pgmap v1362: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 -- 172.21.15.67:0/3105328993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 msgr2=0x7fe378102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 --2- 172.21.15.67:0/3105328993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe378102ff0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802a410 comp rx=0 tx=0).stop 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 -- 172.21.15.67:0/3105328993 shutdown_connections 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 --2- 172.21.15.67:0/3105328993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 0x7fe37810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 --2- 172.21.15.67:0/3105328993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe378108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 --2- 172.21.15.67:0/3105328993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe378102ff0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 -- 172.21.15.67:0/3105328993 >> 172.21.15.67:0/3105328993 conn(0x7fe3780fe660 msgr2=0x7fe378100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:48.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 -- 172.21.15.67:0/3105328993 shutdown_connections 2024-09-10T23:29:48.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.721+0000 7fe37e000640 1 -- 172.21.15.67:0/3105328993 wait complete. 2024-09-10T23:29:48.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.722+0000 7fe37e000640 1 Processor -- start 2024-09-10T23:29:48.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 -- start start 2024-09-10T23:29:48.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:48.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe37819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:48.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 0x7fe3781a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe378112b60 con 0x7fe378102c10 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781129e0 con 0x7fe378108620 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe378112ce0 con 0x7fe378108fa0 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44362/0 (socket says 172.21.15.67:44362) 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.723+0000 7fe37cc00640 1 -- 172.21.15.67:0/2598625336 learned_addr learned my addr 172.21.15.67:0/2598625336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37d600640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 0x7fe3781a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe36fe00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe37819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:48.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 -- 172.21.15.67:0/2598625336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 msgr2=0x7fe3781a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 0x7fe3781a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 -- 172.21.15.67:0/2598625336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 msgr2=0x7fe37819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe37819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 -- 172.21.15.67:0/2598625336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3781a1200 con 0x7fe378102c10 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe36fe00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe37819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:29:48.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe37cc00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe36802a930 tx=0x7fe368003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.724+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36803f070 con 0x7fe378102c10 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.725+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36803baf0 con 0x7fe378102c10 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.725+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3781a1490 con 0x7fe378102c10 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.725+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035030 con 0x7fe378102c10 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.725+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781a7bb0 con 0x7fe378102c10 2024-09-10T23:29:48.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.726+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe378102ff0 con 0x7fe378102c10 2024-09-10T23:29:48.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.728+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe36803b660 con 0x7fe378102c10 2024-09-10T23:29:48.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.729+0000 7fe376000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 0x7fe35c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:48.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.729+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7fe368106d60 con 0x7fe378102c10 2024-09-10T23:29:48.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.730+0000 7fe36fe00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 0x7fe35c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:48.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.730+0000 7fe36fe00640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 0x7fe35c0b79c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe378101f80 tx=0x7fe360005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:48.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.733+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe3680cd4a0 con 0x7fe378102c10 2024-09-10T23:29:48.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.943+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3781a7e40 con 0x7fe378102c10 2024-09-10T23:29:48.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.944+0000 7fe376000640 1 -- 172.21.15.67:0/2598625336 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v137) ==== 81+0+882 (secure 0 0 0) 0x7fe3680ccda0 con 0x7fe378102c10 2024-09-10T23:29:48.945 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2682299290 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:48.945 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3212905705 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/82707279 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3042343451 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3255023915 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2935874097 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:48.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:48.947 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3802899292 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:48.947 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3004982929 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:48.947 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:48.947 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1673435000 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-09-10T23:29:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 msgr2=0x7fe35c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 0x7fe35c0b79c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe378101f80 tx=0x7fe360005f90 comp rx=0 tx=0).stop 2024-09-10T23:29:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 msgr2=0x7fe37819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe36802a930 tx=0x7fe368003b90 comp rx=0 tx=0).stop 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 shutdown_connections 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fe35c0b5500 0x7fe35c0b79c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe378108fa0 0x7fe3781a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe378108620 0x7fe37819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 --2- 172.21.15.67:0/2598625336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378102c10 0x7fe37819b150 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 >> 172.21.15.67:0/2598625336 conn(0x7fe3780fe660 msgr2=0x7fe3780ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.946+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 shutdown_connections 2024-09-10T23:29:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:48.947+0000 7fe37e000640 1 -- 172.21.15.67:0/2598625336 wait complete. 2024-09-10T23:29:49.759 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-10T23:29:49.759 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:29:49.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T23:29:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:49 smithi145 ceph-mon[34397]: pgmap v1363: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:50.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:49 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2598625336' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:50.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:49 smithi092 ceph-mon[34223]: pgmap v1363: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:50.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:49 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2598625336' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:50.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:49 smithi067 ceph-mon[30939]: pgmap v1363: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T23:29:50.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:49 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2598625336' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 -- 172.21.15.67:0/681157816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e4075e00 msgr2=0x7f78e410ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 --2- 172.21.15.67:0/681157816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e4075e00 0x7f78e410ecc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009900 tx=0x7f78d002a410 comp rx=0 tx=0).stop 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 -- 172.21.15.67:0/681157816 shutdown_connections 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 --2- 172.21.15.67:0/681157816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78e410f200 0x7f78e41115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 --2- 172.21.15.67:0/681157816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e4075e00 0x7f78e410ecc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 --2- 172.21.15.67:0/681157816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e40754e0 0x7f78e40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 -- 172.21.15.67:0/681157816 >> 172.21.15.67:0/681157816 conn(0x7f78e40fe660 msgr2=0x7f78e4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 -- 172.21.15.67:0/681157816 shutdown_connections 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.972+0000 7f78ea400640 1 -- 172.21.15.67:0/681157816 wait complete. 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.973+0000 7f78ea400640 1 Processor -- start 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.973+0000 7f78ea400640 1 -- start start 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78e4075e00 0x7f78e419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 0x7f78e41a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4114c90 con 0x7f78e40754e0 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4114b10 con 0x7f78e410f200 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78ea400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4114e10 con 0x7f78e4075e00 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54346/0 (socket says 172.21.15.67:54346) 2024-09-10T23:29:50.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 -- 172.21.15.67:0/1482333503 learned_addr learned my addr 172.21.15.67:0/1482333503 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 -- 172.21.15.67:0/1482333503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78e4075e00 msgr2=0x7f78e419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78e4075e00 0x7f78e419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9a00640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 0x7f78e41a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.974+0000 7f78e9000640 1 -- 172.21.15.67:0/1482333503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 msgr2=0x7f78e41a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e9000640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 0x7f78e41a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e9000640 1 -- 172.21.15.67:0/1482333503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41a1230 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e9a00640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 0x7f78e41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e9000640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e9f0 tx=0x7f78d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d800cd80 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d80108c0 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41a1520 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80187c0 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.975+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41a7be0 con 0x7f78e40754e0 2024-09-10T23:29:50.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.976+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e40758c0 con 0x7f78e40754e0 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.978+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f78d8010430 con 0x7f78e40754e0 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.980+0000 7f78e1600640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 0x7f78cc0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.980+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7f78d80e6900 con 0x7f78e40754e0 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.980+0000 7f78e3e00640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 0x7f78cc0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.981+0000 7f78e3e00640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 0x7f78cc0b79c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005c20 tx=0x7f78d0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:50.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:50.984+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78d80ad040 con 0x7f78e40754e0 2024-09-10T23:29:51.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.194+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f78e41a7fe0 con 0x7f78e40754e0 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.194+0000 7f78e1600640 1 -- 172.21.15.67:0/1482333503 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v137) ==== 81+0+882 (secure 0 0 0) 0x7f78d80ac940 con 0x7f78e40754e0 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2682299290 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3212905705 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/82707279 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3042343451 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:51.196 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3255023915 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2935874097 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/294142637 2024-09-11T22:46:47.684151+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2041533453 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3802899292 2024-09-11T22:47:16.118298+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3004982929 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/363502482 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1673435000 2024-09-11T22:48:31.958130+0000 2024-09-10T23:29:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:listed 15 entries 2024-09-10T23:29:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 msgr2=0x7f78cc0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 0x7f78cc0b79c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005c20 tx=0x7f78d0000bc0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 msgr2=0x7f78e419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e9f0 tx=0x7f78d800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 shutdown_connections 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f78cc0b5500 0x7f78cc0b79c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e410f200 0x7f78e41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78e4075e00 0x7f78e419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 --2- 172.21.15.67:0/1482333503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78e40754e0 0x7f78e419b180 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 >> 172.21.15.67:0/1482333503 conn(0x7f78e40fe660 msgr2=0x7f78e40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 shutdown_connections 2024-09-10T23:29:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:51.197+0000 7f78ea400640 1 -- 172.21.15.67:0/1482333503 wait complete. 2024-09-10T23:29:52.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:51 smithi067 ceph-mon[30939]: pgmap v1364: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:52.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:51 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1482333503' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:52.069 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-09-10T23:29:52.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:29:52.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:29:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:51 smithi145 ceph-mon[34397]: pgmap v1364: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:52.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:51 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1482333503' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:52.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:29:52.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[89085]: fuse finished with error 0 and tester_r 0 2024-09-10T23:29:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:51 smithi092 ceph-mon[34223]: pgmap v1364: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T23:29:52.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:51 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1482333503' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:29:54.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:53 smithi145 ceph-mon[34397]: pgmap v1365: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:54.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:53 smithi092 ceph-mon[34223]: pgmap v1365: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:54.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:53 smithi067 ceph-mon[30939]: pgmap v1365: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T23:29:56.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:55 smithi145 ceph-mon[34397]: pgmap v1366: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:29:56.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:55 smithi092 ceph-mon[34223]: pgmap v1366: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:29:56.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:55 smithi067 ceph-mon[30939]: pgmap v1366: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T23:29:58.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:57 smithi145 ceph-mon[34397]: pgmap v1367: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T23:29:58.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T23:29:58.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:29:58.248 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:29:58.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:57 smithi092 ceph-mon[34223]: pgmap v1367: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T23:29:58.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:29:58.295 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-09-10T23:29:58.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:57 smithi067 ceph-mon[30939]: pgmap v1367: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T23:29:58.335 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:29:58.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:29:58.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:29:58.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:29:58.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-09-10T23:29:58.772 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T23:29:58.782 INFO:tasks.cephadm:Teardown begin 2024-09-10T23:29:58.783 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:29:58.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:29:59.171 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:29:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 -- 172.21.15.67:0/999791619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34102dc0 msgr2=0x7f7c341031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 --2- 172.21.15.67:0/999791619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34102dc0 0x7f7c341031a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009900 tx=0x7f7c2402a410 comp rx=0 tx=0).stop 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 -- 172.21.15.67:0/999791619 shutdown_connections 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 --2- 172.21.15.67:0/999791619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c34109150 0x7f7c3410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 --2- 172.21.15.67:0/999791619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c341087d0 0x7f7c34108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 --2- 172.21.15.67:0/999791619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34102dc0 0x7f7c341031a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 -- 172.21.15.67:0/999791619 >> 172.21.15.67:0/999791619 conn(0x7f7c340fe830 msgr2=0x7f7c34100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 -- 172.21.15.67:0/999791619 shutdown_connections 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.726+0000 7f7c3b400640 1 -- 172.21.15.67:0/999791619 wait complete. 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.727+0000 7f7c3b400640 1 Processor -- start 2024-09-10T23:29:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.727+0000 7f7c3b400640 1 -- start start 2024-09-10T23:29:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c34102dc0 0x7f7c341991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 0x7f7c34199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112d10 con 0x7f7c34109150 2024-09-10T23:29:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112b90 con 0x7f7c34102dc0 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c3b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112e90 con 0x7f7c341087d0 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c39400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c39400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37110/0 (socket says 172.21.15.67:37110) 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c39400640 1 -- 172.21.15.67:0/1769342696 learned_addr learned my addr 172.21.15.67:0/1769342696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c2be00640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 0x7f7c34199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.728+0000 7f7c38a00640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c34102dc0 0x7f7c341991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 -- 172.21.15.67:0/1769342696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 msgr2=0x7f7c34199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 0x7f7c34199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 -- 172.21.15.67:0/1769342696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c34102dc0 msgr2=0x7f7c341991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c34102dc0 0x7f7c341991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 -- 172.21.15.67:0/1769342696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3419f290 con 0x7f7c34109150 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c2be00640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 0x7f7c34199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c39400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00eb10 tx=0x7f7c2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004b20 con 0x7f7c34109150 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c2c004cc0 con 0x7f7c34109150 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c010690 con 0x7f7c34109150 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3419f580 con 0x7f7c34109150 2024-09-10T23:29:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.729+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a5c40 con 0x7f7c34109150 2024-09-10T23:29:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.730+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c341031a0 con 0x7f7c34109150 2024-09-10T23:29:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.733+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7c2c005020 con 0x7f7c34109150 2024-09-10T23:29:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.734+0000 7f7c32000640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 0x7f7c180b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:29:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.734+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7f7c2c0e6910 con 0x7f7c34109150 2024-09-10T23:29:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.734+0000 7f7c38a00640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 0x7f7c180b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:29:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.735+0000 7f7c38a00640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 0x7f7c180b7990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006010 tx=0x7f7c24005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:29:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.738+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7c2c0ad050 con 0x7f7c34109150 2024-09-10T23:29:59.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.970+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c3402e5c0 con 0x7f7c34109150 2024-09-10T23:29:59.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:29:59 smithi067 ceph-mon[30939]: pgmap v1368: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T23:29:59.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.971+0000 7f7c32000640 1 -- 172.21.15.67:0/1769342696 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11910 (secure 0 0 0) 0x7f7c2c0ac950 con 0x7f7c34109150 2024-09-10T23:29:59.973 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:29:59.973 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"btime":"2024-09-10T23:28:38:783141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:28:38.628606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:29:59.973 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 msgr2=0x7f7c180b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 0x7f7c180b7990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006010 tx=0x7f7c24005e80 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 msgr2=0x7f7c3419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00eb10 tx=0x7f7c2c00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 shutdown_connections 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.973+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f7c180b54d0 0x7f7c180b7990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c34109150 0x7f7c3419eac0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c341087d0 0x7f7c34199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 --2- 172.21.15.67:0/1769342696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c34102dc0 0x7f7c341991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 >> 172.21.15.67:0/1769342696 conn(0x7f7c340fe830 msgr2=0x7f7c34100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:29:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 shutdown_connections 2024-09-10T23:29:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:29:59.974+0000 7f7c3b400640 1 -- 172.21.15.67:0/1769342696 wait complete. 2024-09-10T23:30:00.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:29:59 smithi145 ceph-mon[34397]: pgmap v1368: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T23:30:00.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:29:59 smithi092 ceph-mon[34223]: pgmap v1368: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T23:30:00.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:30:01.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:00 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1769342696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:01.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:00 smithi067 ceph-mon[30939]: overall HEALTH_OK 2024-09-10T23:30:01.111 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:00 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1769342696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:01.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:00 smithi145 ceph-mon[34397]: overall HEALTH_OK 2024-09-10T23:30:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:00 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1769342696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:01.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:00 smithi092 ceph-mon[34223]: overall HEALTH_OK 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 -- 172.21.15.67:0/2164534213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108620 msgr2=0x7f54a8108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 --2- 172.21.15.67:0/2164534213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108620 0x7f54a8108a60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5490009900 tx=0x7f549002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 -- 172.21.15.67:0/2164534213 shutdown_connections 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 --2- 172.21.15.67:0/2164534213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8108fa0 0x7f54a810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 --2- 172.21.15.67:0/2164534213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108620 0x7f54a8108a60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 --2- 172.21.15.67:0/2164534213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8102c10 0x7f54a8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 -- 172.21.15.67:0/2164534213 >> 172.21.15.67:0/2164534213 conn(0x7f54a80fe660 msgr2=0x7f54a8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 -- 172.21.15.67:0/2164534213 shutdown_connections 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.683+0000 7f54aca00640 1 -- 172.21.15.67:0/2164534213 wait complete. 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.684+0000 7f54aca00640 1 Processor -- start 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.684+0000 7f54aca00640 1 -- start start 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8102c10 0x7f54a8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8108620 0x7f54a81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112b10 con 0x7f54a8108fa0 2024-09-10T23:30:01.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112990 con 0x7f54a8108620 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54aca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112c90 con 0x7f54a8102c10 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37132/0 (socket says 172.21.15.67:37132) 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 -- 172.21.15.67:0/1846802573 learned_addr learned my addr 172.21.15.67:0/1846802573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f549ea00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8108620 0x7f54a81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7400640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8102c10 0x7f54a8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 -- 172.21.15.67:0/1846802573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8102c10 msgr2=0x7f54a8198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8102c10 0x7f54a8198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 -- 172.21.15.67:0/1846802573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8108620 msgr2=0x7f54a81994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.685+0000 7f54a7e00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8108620 0x7f54a81994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54a7e00640 1 -- 172.21.15.67:0/1846802573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a819f010 con 0x7f54a8108fa0 2024-09-10T23:30:01.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54a7e00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f54a000eaa0 tx=0x7f54a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a000cdb0 con 0x7f54a8108fa0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a0005020 con 0x7f54a8108fa0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a819f300 con 0x7f54a8108fa0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00116d0 con 0x7f54a8108fa0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.686+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a59c0 con 0x7f54a8108fa0 2024-09-10T23:30:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.687+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a8102ff0 con 0x7f54a8108fa0 2024-09-10T23:30:01.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.690+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f54a000f040 con 0x7f54a8108fa0 2024-09-10T23:30:01.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.691+0000 7f54a4c00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 0x7f548c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:01.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.691+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7f54a00e6590 con 0x7f54a8108fa0 2024-09-10T23:30:01.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.691+0000 7f54a7400640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 0x7f548c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:01.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.692+0000 7f54a7400640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 0x7f548c0b79c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5498009bd0 tx=0x7f5498005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:01.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.693+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f54a0016030 con 0x7f54a8108fa0 2024-09-10T23:30:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.928+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54a819a090 con 0x7f54a8108fa0 2024-09-10T23:30:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.928+0000 7f54a4c00640 1 -- 172.21.15.67:0/1846802573 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11910 (secure 0 0 0) 0x7f54a00c5ea0 con 0x7f54a8108fa0 2024-09-10T23:30:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:30:01.930 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"btime":"2024-09-10T23:28:38:783141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/99494350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":99494350},{"type":"v1","addr":"172.21.15.67:6835","nonce":99494350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1279733318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1279733318},{"type":"v1","addr":"172.21.15.67:6837","nonce":1279733318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14802,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2446174169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2446174169},{"type":"v1","addr":"172.21.15.92:6835","nonce":2446174169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14853,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/4128597236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":4128597236},{"type":"v1","addr":"172.21.15.67:6839","nonce":4128597236}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/2410494517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":2410494517},{"type":"v1","addr":"172.21.15.67:6841","nonce":2410494517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/482762321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":482762321},{"type":"v1","addr":"172.21.15.145:6833","nonce":482762321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:56:19.886591+0000","modified":"2024-09-10T23:28:38.628606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34730,"mds_1":34691,"mds_2":34631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24664":{"gid":24664,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/2757139462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2757139462},{"type":"v1","addr":"172.21.15.145:6835","nonce":2757139462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/1285378679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1285378679},{"type":"v1","addr":"172.21.15.145:6837","nonce":1285378679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24691":{"gid":24691,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/4174586003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":4174586003},{"type":"v1","addr":"172.21.15.145:6839","nonce":4174586003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34631":{"gid":34631,"name":"k","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.92:6833/307002386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":307002386},{"type":"v1","addr":"172.21.15.92:6833","nonce":307002386}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34691":{"gid":34691,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.92:6837/523163273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":523163273},{"type":"v1","addr":"172.21.15.92:6837","nonce":523163273}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34730":{"gid":34730,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/1949986898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1949986898},{"type":"v1","addr":"172.21.15.92:6839","nonce":1949986898}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34730,"qdb_cluster":[34631,34691,34730]},"id":1}]} 2024-09-10T23:30:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-09-10T23:30:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 msgr2=0x7f548c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 0x7f548c0b79c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5498009bd0 tx=0x7f5498005f90 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 msgr2=0x7f54a819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f54a000eaa0 tx=0x7f54a000ef60 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 shutdown_connections 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f548c0b5500 0x7f548c0b79c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54a8108fa0 0x7f54a819e840 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54a8108620 0x7f54a81994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 --2- 172.21.15.67:0/1846802573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54a8102c10 0x7f54a8198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 >> 172.21.15.67:0/1846802573 conn(0x7f54a80fe660 msgr2=0x7f54a8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 shutdown_connections 2024-09-10T23:30:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:01.931+0000 7f54aca00640 1 -- 172.21.15.67:0/1846802573 wait complete. 2024-09-10T23:30:02.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:01 smithi145 ceph-mon[34397]: pgmap v1369: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T23:30:02.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:01 smithi092 ceph-mon[34223]: pgmap v1369: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T23:30:02.281 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:01 smithi067 ceph-mon[30939]: pgmap v1369: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T23:30:02.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T23:30:03.037 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:03.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:02 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1846802573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:03.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:02 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1846802573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:03.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:02 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1846802573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 -- 172.21.15.67:0/3672559561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 msgr2=0x7fa98c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 --2- 172.21.15.67:0/3672559561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c105380 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009900 tx=0x7fa97c02a410 comp rx=0 tx=0).stop 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 -- 172.21.15.67:0/3672559561 shutdown_connections 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 --2- 172.21.15.67:0/3672559561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 0x7fa98c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 --2- 172.21.15.67:0/3672559561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 0x7fa98c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 --2- 172.21.15.67:0/3672559561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c105380 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 -- 172.21.15.67:0/3672559561 >> 172.21.15.67:0/3672559561 conn(0x7fa98c100a50 msgr2=0x7fa98c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 -- 172.21.15.67:0/3672559561 shutdown_connections 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.598+0000 7fa993000640 1 -- 172.21.15.67:0/3672559561 wait complete. 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 Processor -- start 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 -- start start 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 0x7fa98c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 0x7fa98c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c114e10 con 0x7fa98c104fa0 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c114c90 con 0x7fa98c10a9b0 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.599+0000 7fa993000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c114f90 con 0x7fa98c10b330 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37144/0 (socket says 172.21.15.67:37144) 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 -- 172.21.15.67:0/173124832 learned_addr learned my addr 172.21.15.67:0/173124832 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98b400640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 0x7fa98c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa991000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 0x7fa98c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 -- 172.21.15.67:0/173124832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 msgr2=0x7fa98c1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 0x7fa98c1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 -- 172.21.15.67:0/173124832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 msgr2=0x7fa98c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 0x7fa98c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.600+0000 7fa98be00640 1 -- 172.21.15.67:0/173124832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c1a14d0 con 0x7fa98c104fa0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa98be00640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009230 tx=0x7fa97c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c03f030 con 0x7fa98c104fa0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97c003a80 con 0x7fa98c104fa0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c1a1760 con 0x7fa98c104fa0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c035070 con 0x7fa98c104fa0 2024-09-10T23:30:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.601+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a7e80 con 0x7fa98c104fa0 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.603+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c105380 con 0x7fa98c104fa0 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.605+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa97c03bc30 con 0x7fa98c104fa0 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.606+0000 7fa988c00640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 0x7fa9700b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.607+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7fa97c106d50 con 0x7fa98c104fa0 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.607+0000 7fa98b400640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 0x7fa9700b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.608+0000 7fa98b400640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 0x7fa9700b7990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa98c10a670 tx=0x7fa97800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.611+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa97c0cce40 con 0x7fa98c104fa0 2024-09-10T23:30:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.811+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa98c10cb80 con 0x7fa98c104fa0 2024-09-10T23:30:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.813+0000 7fa988c00640 1 -- 172.21.15.67:0/173124832 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+25409 (secure 0 0 0) 0x7fa97c0d1870 con 0x7fa98c104fa0 2024-09-10T23:30:03.815 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-10T23:30:03.815 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":137,"fsid":"5c4a3ee4-6fc6-11ef-bcea-c7b262605968","created":"2024-09-10T22:46:22.380818+0000","modified":"2024-09-10T23:29:19.052293+0000","last_up_change":"2024-09-10T22:55:17.387981+0000","last_in_change":"2024-09-10T22:54:50.219954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:50:44.441563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:56:14.927478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:56:17.858506+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:56:25.944001+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7bebf858-8d3f-46bb-aea6-043414353290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6803","nonce":3409065493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6805","nonce":3409065493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6809","nonce":3409065493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3409065493},{"type":"v1","addr":"172.21.15.67:6807","nonce":3409065493}]},"public_addr":"172.21.15.67:6803/3409065493","cluster_addr":"172.21.15.67:6805/3409065493","heartbeat_back_addr":"172.21.15.67:6809/3409065493","heartbeat_front_addr":"172.21.15.67:6807/3409065493","state":["exists","up"]},{"osd":1,"uuid":"002adf57-737e-4e91-9cf5-164895bffa8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6801","nonce":1917308558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6803","nonce":1917308558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6807","nonce":1917308558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1917308558},{"type":"v1","addr":"172.21.15.92:6805","nonce":1917308558}]},"public_addr":"172.21.15.92:6801/1917308558","cluster_addr":"172.21.15.92:6803/1917308558","heartbeat_back_addr":"172.21.15.92:6807/1917308558","heartbeat_front_addr":"172.21.15.92:6805/1917308558","state":["exists","up"]},{"osd":2,"uuid":"210c2de7-0ad3-4e05-b098-446542372b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6801","nonce":596498594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6803","nonce":596498594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6807","nonce":596498594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":596498594},{"type":"v1","addr":"172.21.15.145:6805","nonce":596498594}]},"public_addr":"172.21.15.145:6801/596498594","cluster_addr":"172.21.15.145:6803/596498594","heartbeat_back_addr":"172.21.15.145:6807/596498594","heartbeat_front_addr":"172.21.15.145:6805/596498594","state":["exists","up"]},{"osd":3,"uuid":"50b25a5d-e7a9-448a-b4ae-435368940e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6811","nonce":3883722074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6813","nonce":3883722074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6817","nonce":3883722074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3883722074},{"type":"v1","addr":"172.21.15.67:6815","nonce":3883722074}]},"public_addr":"172.21.15.67:6811/3883722074","cluster_addr":"172.21.15.67:6813/3883722074","heartbeat_back_addr":"172.21.15.67:6817/3883722074","heartbeat_front_addr":"172.21.15.67:6815/3883722074","state":["exists","up"]},{"osd":4,"uuid":"18c7f079-20a7-4726-bf63-797bd66fc512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6809","nonce":1079057674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6811","nonce":1079057674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6815","nonce":1079057674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1079057674},{"type":"v1","addr":"172.21.15.92:6813","nonce":1079057674}]},"public_addr":"172.21.15.92:6809/1079057674","cluster_addr":"172.21.15.92:6811/1079057674","heartbeat_back_addr":"172.21.15.92:6815/1079057674","heartbeat_front_addr":"172.21.15.92:6813/1079057674","state":["exists","up"]},{"osd":5,"uuid":"8c623a06-702d-41dc-a7a7-56bc739d3199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6809","nonce":510004066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6811","nonce":510004066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6815","nonce":510004066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":510004066},{"type":"v1","addr":"172.21.15.145:6813","nonce":510004066}]},"public_addr":"172.21.15.145:6809/510004066","cluster_addr":"172.21.15.145:6811/510004066","heartbeat_back_addr":"172.21.15.145:6815/510004066","heartbeat_front_addr":"172.21.15.145:6813/510004066","state":["exists","up"]},{"osd":6,"uuid":"9ca47089-cde1-479f-95ed-70c93dfafcf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6819","nonce":2528830633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6821","nonce":2528830633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6825","nonce":2528830633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2528830633},{"type":"v1","addr":"172.21.15.67:6823","nonce":2528830633}]},"public_addr":"172.21.15.67:6819/2528830633","cluster_addr":"172.21.15.67:6821/2528830633","heartbeat_back_addr":"172.21.15.67:6825/2528830633","heartbeat_front_addr":"172.21.15.67:6823/2528830633","state":["exists","up"]},{"osd":7,"uuid":"726d7795-37f9-452e-a9a0-d764920275a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6817","nonce":3770526101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6819","nonce":3770526101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6823","nonce":3770526101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3770526101},{"type":"v1","addr":"172.21.15.92:6821","nonce":3770526101}]},"public_addr":"172.21.15.92:6817/3770526101","cluster_addr":"172.21.15.92:6819/3770526101","heartbeat_back_addr":"172.21.15.92:6823/3770526101","heartbeat_front_addr":"172.21.15.92:6821/3770526101","state":["exists","up"]},{"osd":8,"uuid":"f8630c3d-7812-4bca-a22d-3539f4fa03d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6817","nonce":1197192386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6819","nonce":1197192386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6823","nonce":1197192386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1197192386},{"type":"v1","addr":"172.21.15.145:6821","nonce":1197192386}]},"public_addr":"172.21.15.145:6817/1197192386","cluster_addr":"172.21.15.145:6819/1197192386","heartbeat_back_addr":"172.21.15.145:6823/1197192386","heartbeat_front_addr":"172.21.15.145:6821/1197192386","state":["exists","up"]},{"osd":9,"uuid":"c52ab99b-433a-44a6-9fab-dbd35f1bf6d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6827","nonce":3621288018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6829","nonce":3621288018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6833","nonce":3621288018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3621288018},{"type":"v1","addr":"172.21.15.67:6831","nonce":3621288018}]},"public_addr":"172.21.15.67:6827/3621288018","cluster_addr":"172.21.15.67:6829/3621288018","heartbeat_back_addr":"172.21.15.67:6833/3621288018","heartbeat_front_addr":"172.21.15.67:6831/3621288018","state":["exists","up"]},{"osd":10,"uuid":"e96f60cf-4ed2-4c62-bdfd-90c1632a5632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6825","nonce":1466235369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6827","nonce":1466235369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6831","nonce":1466235369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1466235369},{"type":"v1","addr":"172.21.15.92:6829","nonce":1466235369}]},"public_addr":"172.21.15.92:6825/1466235369","cluster_addr":"172.21.15.92:6827/1466235369","heartbeat_back_addr":"172.21.15.92:6831/1466235369","heartbeat_front_addr":"172.21.15.92:6829/1466235369","state":["exists","up"]},{"osd":11,"uuid":"d3c1830f-3c31-4764-92c2-ffcd6c11acce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6825","nonce":257658495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6827","nonce":257658495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6831","nonce":257658495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":257658495},{"type":"v1","addr":"172.21.15.145:6829","nonce":257658495}]},"public_addr":"172.21.15.145:6825/257658495","cluster_addr":"172.21.15.145:6827/257658495","heartbeat_back_addr":"172.21.15.145:6831/257658495","heartbeat_front_addr":"172.21.15.145:6829/257658495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:41.541374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:10.801963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:50:40.843621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:11.399642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:51:41.576144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:11.099855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:52:42.373108+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:12.526158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:53:42.737487+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:13.941663+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:54:43.463548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:55:15.037953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2682299290":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6801/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3212905705":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/82707279":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/3042343451":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3255023915":"2024-09-11T22:46:47.684151+0000","172.21.15.67:0/2935874097":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6800/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/294142637":"2024-09-11T22:46:47.684151+0000","172.21.15.67:6801/2041533453":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3802899292":"2024-09-11T22:47:16.118298+0000","172.21.15.67:0/3004982929":"2024-09-11T22:48:31.958130+0000","172.21.15.67:6800/363502482":"2024-09-11T22:48:31.958130+0000","172.21.15.67:0/1673435000":"2024-09-11T22:48:31.958130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]},{"pool":4,"snaps":[{"begin":42,"length":1},{"begin":55,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 msgr2=0x7fa9700b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 0x7fa9700b7990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa98c10a670 tx=0x7fa97800b410 comp rx=0 tx=0).stop 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 msgr2=0x7fa98c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009230 tx=0x7fa97c004270 comp rx=0 tx=0).stop 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 shutdown_connections 2024-09-10T23:30:03.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7fa9700b54d0 0x7fa9700b7990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa98c10b330 0x7fa98c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa98c10a9b0 0x7fa98c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 --2- 172.21.15.67:0/173124832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98c104fa0 0x7fa98c19b420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 >> 172.21.15.67:0/173124832 conn(0x7fa98c100a50 msgr2=0x7fa98c101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 shutdown_connections 2024-09-10T23:30:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:03.816+0000 7fa993000640 1 -- 172.21.15.67:0/173124832 wait complete. 2024-09-10T23:30:04.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:03 smithi145 ceph-mon[34397]: pgmap v1370: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-10T23:30:04.244 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:03 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/173124832' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:30:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:03 smithi067 ceph-mon[30939]: pgmap v1370: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-10T23:30:04.245 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:03 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/173124832' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:30:04.245 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:03 smithi092 ceph-mon[34223]: pgmap v1370: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-10T23:30:04.245 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:03 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/173124832' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:30:04.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T23:30:04.925 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:05.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:04 smithi145 ceph-mon[34397]: pgmap v1371: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-10T23:30:05.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:04 smithi067 ceph-mon[30939]: pgmap v1371: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-10T23:30:05.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:04 smithi092 ceph-mon[34223]: pgmap v1371: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 -- 172.21.15.67:0/2181895474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d4104e10 msgr2=0x7f65d41051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 --2- 172.21.15.67:0/2181895474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d4104e10 0x7f65d41051f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009900 tx=0x7f65c402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 -- 172.21.15.67:0/2181895474 shutdown_connections 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 --2- 172.21.15.67:0/2181895474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d410b1a0 0x7f65d41119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 --2- 172.21.15.67:0/2181895474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410a820 0x7f65d410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 --2- 172.21.15.67:0/2181895474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d4104e10 0x7f65d41051f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 -- 172.21.15.67:0/2181895474 >> 172.21.15.67:0/2181895474 conn(0x7f65d4100840 msgr2=0x7f65d4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:05.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 -- 172.21.15.67:0/2181895474 shutdown_connections 2024-09-10T23:30:05.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.496+0000 7f65d3200640 1 -- 172.21.15.67:0/2181895474 wait complete. 2024-09-10T23:30:05.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.497+0000 7f65d3200640 1 Processor -- start 2024-09-10T23:30:05.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 -- start start 2024-09-10T23:30:05.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d4104e10 0x7f65d419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:05.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410b1a0 0x7f65d41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4114d60 con 0x7f65d410a820 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4114be0 con 0x7f65d410b1a0 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4114ee0 con 0x7f65d4104e10 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37154/0 (socket says 172.21.15.67:37154) 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d1400640 1 -- 172.21.15.67:0/2301246751 learned_addr learned my addr 172.21.15.67:0/2301246751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d1e00640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d4104e10 0x7f65d419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.498+0000 7f65d2800640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410b1a0 0x7f65d41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 -- 172.21.15.67:0/2301246751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d4104e10 msgr2=0x7f65d419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d4104e10 0x7f65d419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 -- 172.21.15.67:0/2301246751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410b1a0 msgr2=0x7f65d41a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:05.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410b1a0 0x7f65d41a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 -- 172.21.15.67:0/2301246751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d41a1240 con 0x7f65d410a820 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d1400640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e9f0 tx=0x7f65c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c000cd80 con 0x7f65d410a820 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c00108c0 con 0x7f65d410a820 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d41a1530 con 0x7f65d410a820 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00187c0 con 0x7f65d410a820 2024-09-10T23:30:05.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.499+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d41a7bf0 con 0x7f65d410a820 2024-09-10T23:30:05.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.501+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d41051f0 con 0x7f65d410a820 2024-09-10T23:30:05.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.504+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f65c0010430 con 0x7f65d410a820 2024-09-10T23:30:05.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.505+0000 7f65caa00640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 0x7f65b80b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:05.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.505+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9519+0+0 (secure 0 0 0) 0x7f65c00e6cd0 con 0x7f65d410a820 2024-09-10T23:30:05.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.505+0000 7f65d1e00640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 0x7f65b80b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:05.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.506+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65c00ad410 con 0x7f65d410a820 2024-09-10T23:30:05.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.506+0000 7f65d1e00640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 0x7f65b80b79c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c4005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:05.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.746+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f65d419c2c0 con 0x7f65d410a820 2024-09-10T23:30:05.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.767+0000 7f65caa00640 1 -- 172.21.15.67:0/2301246751 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f65d419c2c0 con 0x7f65d410a820 2024-09-10T23:30:05.768 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:30:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.771+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 msgr2=0x7f65b80b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.771+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 0x7f65b80b79c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c4005e80 comp rx=0 tx=0).stop 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.771+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 msgr2=0x7f65d419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.771+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e9f0 tx=0x7f65c000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 shutdown_connections 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f65b80b5500 0x7f65b80b79c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65d410b1a0 0x7f65d41a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d410a820 0x7f65d419b6d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 --2- 172.21.15.67:0/2301246751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65d4104e10 0x7f65d419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.772+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 >> 172.21.15.67:0/2301246751 conn(0x7f65d4100840 msgr2=0x7f65d4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.773+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 shutdown_connections 2024-09-10T23:30:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:05.773+0000 7f65d3200640 1 -- 172.21.15.67:0/2301246751 wait complete. 2024-09-10T23:30:06.033 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: osdmap e138: 12 total, 12 up, 12 in 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mon[34223]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.034 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mds[65474]: -- 172.21.15.92:0/1593550541 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c75379c20 con 0x564c75356800 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: starting mds.b at 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-mds[65173]: -- 172.21.15.92:0/1232834802 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635a5031860 con 0x5635a500f400 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: starting mds.h at 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.035 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.036 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:30:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: starting mds.k at 2024-09-10T23:30:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a[30935]: 2024-09-10T23:30:05.765+0000 7f5baf200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:30:06.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:30:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:30:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:30:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:30:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: osdmap e138: 12 total, 12 up, 12 in 2024-09-10T23:30:06.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:05 smithi067 ceph-mon[30939]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T23:30:06.130 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.130 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.130 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mds[66384]: -- 172.21.15.145:0/395196930 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c681a5860 con 0x556c68182800 2024-09-10T23:30:06.130 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: starting mds.c at 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2301246751' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: osdmap e138: 12 total, 12 up, 12 in 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mon[34397]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.131 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mds[66683]: -- 172.21.15.145:0/668691585 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d24bf7860 con 0x561d24bd4800 2024-09-10T23:30:06.132 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: starting mds.i at 2024-09-10T23:30:06.132 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f[66082]: ignoring --setuser ceph since I am not root 2024-09-10T23:30:06.132 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f[66082]: ignoring --setgroup ceph since I am not root 2024-09-10T23:30:06.132 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-mds[66086]: -- 172.21.15.145:0/473559652 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f49e9bc20 con 0x559f49e79000 2024-09-10T23:30:06.132 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:30:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f[66082]: starting mds.f at 2024-09-10T23:30:06.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:30:06.918 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 -- 172.21.15.67:0/2383865282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 msgr2=0x7f054410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 --2- 172.21.15.67:0/2383865282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054410ade0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02a410 comp rx=0 tx=0).stop 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 -- 172.21.15.67:0/2383865282 shutdown_connections 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 --2- 172.21.15.67:0/2383865282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 0x7f0544111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 --2- 172.21.15.67:0/2383865282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054410ade0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 --2- 172.21.15.67:0/2383865282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f0544105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 -- 172.21.15.67:0/2383865282 >> 172.21.15.67:0/2383865282 conn(0x7f0544100a00 msgr2=0x7f0544102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 -- 172.21.15.67:0/2383865282 shutdown_connections 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.474+0000 7f0548a00640 1 -- 172.21.15.67:0/2383865282 wait complete. 2024-09-10T23:30:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 Processor -- start 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 -- start start 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f054419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 0x7f05441a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0544114df0 con 0x7f054410a9a0 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0544114c70 con 0x7f054410b320 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.475+0000 7f0548a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0544114f70 con 0x7f0544104f90 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37180/0 (socket says 172.21.15.67:37180) 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 -- 172.21.15.67:0/2594907097 learned_addr learned my addr 172.21.15.67:0/2594907097 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0542800640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 0x7f05441a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0541e00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f054419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 -- 172.21.15.67:0/2594907097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 msgr2=0x7f054419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f054419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 -- 172.21.15.67:0/2594907097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 msgr2=0x7f05441a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 0x7f05441a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0539400640 1 -- 172.21.15.67:0/2594907097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05441a1450 con 0x7f054410a9a0 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.476+0000 7f0541e00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f054419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:30:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f0539400640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c03f030 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05441a16e0 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f052c003a80 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c035070 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.477+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05441a7e00 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.478+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0544105370 con 0x7f054410a9a0 2024-09-10T23:30:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.481+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f052c03bc30 con 0x7f054410a9a0 2024-09-10T23:30:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.482+0000 7f053b400640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 0x7f05280b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.482+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9689+0+0 (secure 0 0 0) 0x7f052c1073f0 con 0x7f054410a9a0 2024-09-10T23:30:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.482+0000 7f0541e00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 0x7f05280b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.483+0000 7f0541e00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 0x7f05280b7910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0534009b40 tx=0x7f0534005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.484+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f052c0cda80 con 0x7f054410a9a0 2024-09-10T23:30:07.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.717+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f054410cb70 con 0x7f054410a9a0 2024-09-10T23:30:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.882+0000 7f053b400640 1 -- 172.21.15.67:0/2594907097 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f052c0d6230 con 0x7f054410a9a0 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.884+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 msgr2=0x7f05280b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 0x7f05280b7910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0534009b40 tx=0x7f0534005e50 comp rx=0 tx=0).stop 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 msgr2=0x7f054419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c004270 comp rx=0 tx=0).stop 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 shutdown_connections 2024-09-10T23:30:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f05280b5450 0x7f05280b7910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f054410b320 0x7f05441a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f054410a9a0 0x7f054419b8e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 --2- 172.21.15.67:0/2594907097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0544104f90 0x7f054419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 >> 172.21.15.67:0/2594907097 conn(0x7f0544100a00 msgr2=0x7f0544102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 shutdown_connections 2024-09-10T23:30:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:07.885+0000 7f0548a00640 1 -- 172.21.15.67:0/2594907097 wait complete. 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: pgmap v1373: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6834/2339696456,v1:172.21.15.145:6836/2339696456] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6835/2194345624,v1:172.21.15.145:6837/2194345624] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6832/3228534836,v1:172.21.15.92:6833/3228534836] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.145:6838/1347717611,v1:172.21.15.145:6839/1347717611] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6836/1454141189,v1:172.21.15.92:6837/1454141189] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: mds.? [v2:172.21.15.92:6838/4199599283,v1:172.21.15.92:6839/4199599283] up:boot 2024-09-10T23:30:08.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:30:08.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:07 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: pgmap v1373: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6834/2339696456,v1:172.21.15.145:6836/2339696456] up:boot 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6835/2194345624,v1:172.21.15.145:6837/2194345624] up:boot 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6832/3228534836,v1:172.21.15.92:6833/3228534836] up:boot 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.145:6838/1347717611,v1:172.21.15.145:6839/1347717611] up:boot 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6836/1454141189,v1:172.21.15.92:6837/1454141189] up:boot 2024-09-10T23:30:08.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: mds.? [v2:172.21.15.92:6838/4199599283,v1:172.21.15.92:6839/4199599283] up:boot 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:30:08.131 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:07 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: pgmap v1373: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6834/2339696456,v1:172.21.15.145:6836/2339696456] up:boot 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6835/2194345624,v1:172.21.15.145:6837/2194345624] up:boot 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6832/3228534836,v1:172.21.15.92:6833/3228534836] up:boot 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.145:6838/1347717611,v1:172.21.15.145:6839/1347717611] up:boot 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6836/1454141189,v1:172.21.15.92:6837/1454141189] up:boot 2024-09-10T23:30:08.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: mds.? [v2:172.21.15.92:6838/4199599283,v1:172.21.15.92:6839/4199599283] up:boot 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='mgr.14223 172.21.15.67:0/2925781126' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:30:08.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:07 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:30:08.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:30:08.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:08 smithi067 ceph-mon[30939]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:30:08.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:08 smithi067 ceph-mon[30939]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:30:08.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:08 smithi067 ceph-mon[30939]: Cluster is now healthy 2024-09-10T23:30:08.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:08 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:30:08.959 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:08 smithi067 ceph-mon[30939]: fsmap 12 up:standby 2024-09-10T23:30:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:08 smithi145 ceph-mon[34397]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:30:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:08 smithi145 ceph-mon[34397]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:30:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:08 smithi145 ceph-mon[34397]: Cluster is now healthy 2024-09-10T23:30:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:08 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:30:09.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:08 smithi145 ceph-mon[34397]: fsmap 12 up:standby 2024-09-10T23:30:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:08 smithi092 ceph-mon[34223]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:30:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:08 smithi092 ceph-mon[34223]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:30:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:08 smithi092 ceph-mon[34223]: Cluster is now healthy 2024-09-10T23:30:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:08 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2594907097' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:30:09.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:08 smithi092 ceph-mon[34223]: fsmap 12 up:standby 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 -- 172.21.15.67:0/2527674804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 msgr2=0x7f9fe810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2527674804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe810f7a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009900 tx=0x7f9fe002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 -- 172.21.15.67:0/2527674804 shutdown_connections 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2527674804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe810f7a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2527674804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8108620 0x7f9fe8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2527674804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8102c10 0x7f9fe8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 -- 172.21.15.67:0/2527674804 >> 172.21.15.67:0/2527674804 conn(0x7f9fe80fe660 msgr2=0x7f9fe8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:09.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 -- 172.21.15.67:0/2527674804 shutdown_connections 2024-09-10T23:30:09.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.502+0000 7f9fe7200640 1 -- 172.21.15.67:0/2527674804 wait complete. 2024-09-10T23:30:09.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 Processor -- start 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 -- start start 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102c10 0x7f9fe819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8108620 0x7f9fe819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8112b20 con 0x7f9fe8108fa0 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81129a0 con 0x7f9fe8108620 2024-09-10T23:30:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.503+0000 7f9fe7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8112ca0 con 0x7f9fe8102c10 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53388/0 (socket says 172.21.15.67:53388) 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 -- 172.21.15.67:0/2415133650 learned_addr learned my addr 172.21.15.67:0/2415133650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe5e00640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102c10 0x7f9fe819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fdd400640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8108620 0x7f9fe819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 -- 172.21.15.67:0/2415133650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102c10 msgr2=0x7f9fe819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102c10 0x7f9fe819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 -- 172.21.15.67:0/2415133650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8108620 msgr2=0x7f9fe819b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8108620 0x7f9fe819b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.504+0000 7f9fe6800640 1 -- 172.21.15.67:0/2415133650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81a11c0 con 0x7f9fe8108fa0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fe6800640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004770 tx=0x7f9fe0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe003f070 con 0x7f9fe8108fa0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fe003bd00 con 0x7f9fe8108fa0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81a1450 con 0x7f9fe8108fa0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0035030 con 0x7f9fe8108fa0 2024-09-10T23:30:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.505+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81a7b70 con 0x7f9fe8108fa0 2024-09-10T23:30:09.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.507+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe8102ff0 con 0x7f9fe8108fa0 2024-09-10T23:30:09.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.509+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9fe0031be0 con 0x7f9fe8108fa0 2024-09-10T23:30:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.510+0000 7f9fdf400640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 0x7f9fcc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.510+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9689+0+0 (secure 0 0 0) 0x7f9fe0106e70 con 0x7f9fe8108fa0 2024-09-10T23:30:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.510+0000 7f9fe5e00640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 0x7f9fcc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.511+0000 7f9fe5e00640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 0x7f9fcc0b7a40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8005fd0 tx=0x7f9fd8009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.514+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9fe00cceb0 con 0x7f9fe8108fa0 2024-09-10T23:30:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.711+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fe81a7e30 con 0x7f9fe8108fa0 2024-09-10T23:30:09.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.886+0000 7f9fdf400640 1 -- 172.21.15.67:0/2415133650 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) ==== 181+0+0 (secure 0 0 0) 0x7f9fe00d18e0 con 0x7f9fe8108fa0 2024-09-10T23:30:09.887 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:30:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.889+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 msgr2=0x7f9fcc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.889+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 0x7f9fcc0b7a40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8005fd0 tx=0x7f9fd8009680 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.889+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 msgr2=0x7f9fe81a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.889+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004770 tx=0x7f9fe0003cd0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 shutdown_connections 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f9fcc0b5580 0x7f9fcc0b7a40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8108fa0 0x7f9fe81a09f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fe8108620 0x7f9fe819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 --2- 172.21.15.67:0/2415133650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9fe8102c10 0x7f9fe819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.890+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 >> 172.21.15.67:0/2415133650 conn(0x7f9fe80fe660 msgr2=0x7f9fe80753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.891+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 shutdown_connections 2024-09-10T23:30:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:09.891+0000 7f9fe7200640 1 -- 172.21.15.67:0/2415133650 wait complete. 2024-09-10T23:30:10.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:09 smithi067 ceph-mon[30939]: pgmap v1374: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-10T23:30:10.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:09 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:09 smithi145 ceph-mon[34397]: pgmap v1374: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-10T23:30:10.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:09 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:10.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:09 smithi092 ceph-mon[34223]: pgmap v1374: 129 pgs: 129 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-10T23:30:10.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:09 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:10.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:30:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:11.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:10 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:11.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:10 smithi067 ceph-mon[30939]: osdmap e139: 12 total, 12 up, 12 in 2024-09-10T23:30:11.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:10 smithi067 ceph-mon[30939]: pgmap v1376: 97 pgs: 97 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:11.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:10 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:11.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:10 smithi092 ceph-mon[34223]: osdmap e139: 12 total, 12 up, 12 in 2024-09-10T23:30:11.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:10 smithi092 ceph-mon[34223]: pgmap v1376: 97 pgs: 97 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:10 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/2415133650' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:10 smithi145 ceph-mon[34397]: osdmap e139: 12 total, 12 up, 12 in 2024-09-10T23:30:11.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:10 smithi145 ceph-mon[34397]: pgmap v1376: 97 pgs: 97 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 -- 172.21.15.67:0/706787082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8500104fc0 msgr2=0x7f85001053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 --2- 172.21.15.67:0/706787082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8500104fc0 0x7f85001053a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009ca0 tx=0x7f84f002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 -- 172.21.15.67:0/706787082 shutdown_connections 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 --2- 172.21.15.67:0/706787082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 0x7f8500111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 --2- 172.21.15.67:0/706787082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850010a9d0 0x7f850010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 --2- 172.21.15.67:0/706787082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8500104fc0 0x7f85001053a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 -- 172.21.15.67:0/706787082 >> 172.21.15.67:0/706787082 conn(0x7f85001009f0 msgr2=0x7f8500102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 -- 172.21.15.67:0/706787082 shutdown_connections 2024-09-10T23:30:11.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.561+0000 7f8505000640 1 -- 172.21.15.67:0/706787082 wait complete. 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.562+0000 7f8505000640 1 Processor -- start 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.562+0000 7f8505000640 1 -- start start 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500104fc0 0x7f850019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 0x7f85001a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8500114e80 con 0x7f850010a9d0 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8500114d00 con 0x7f850010b350 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f8505000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8500115000 con 0x7f8500104fc0 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f84f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f84f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53404/0 (socket says 172.21.15.67:53404) 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f84f5400640 1 -- 172.21.15.67:0/1479045489 learned_addr learned my addr 172.21.15.67:0/1479045489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f84fde00640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500104fc0 0x7f850019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.563+0000 7f84fe800640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 0x7f85001a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 -- 172.21.15.67:0/1479045489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500104fc0 msgr2=0x7f850019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500104fc0 0x7f850019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 -- 172.21.15.67:0/1479045489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 msgr2=0x7f85001a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 0x7f85001a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 -- 172.21.15.67:0/1479045489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a14e0 con 0x7f850010a9d0 2024-09-10T23:30:11.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f5400640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e800d770 tx=0x7f84e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8017070 con 0x7f850010a9d0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84e80054b0 con 0x7f850010a9d0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a17d0 con 0x7f850010a9d0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8004c60 con 0x7f850010a9d0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.564+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a7e90 con 0x7f850010a9d0 2024-09-10T23:30:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.566+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85001053a0 con 0x7f850010a9d0 2024-09-10T23:30:11.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.568+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f84e8004e00 con 0x7f850010a9d0 2024-09-10T23:30:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.570+0000 7f84f7400640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 0x7f84e40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.570+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9156+0+0 (secure 0 0 0) 0x7f84e80e5c20 con 0x7f850010a9d0 2024-09-10T23:30:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.570+0000 7f84fde00640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 0x7f84e40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:11.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.571+0000 7f84fde00640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 0x7f84e40b7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f84f0000f80 tx=0x7f84f0031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.573+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f84e80abe70 con 0x7f850010a9d0 2024-09-10T23:30:11.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.768+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f850010d190 con 0x7f850010a9d0 2024-09-10T23:30:11.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.901+0000 7f84f7400640 1 -- 172.21.15.67:0/1479045489 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) ==== 169+0+0 (secure 0 0 0) 0x7f84e80b08a0 con 0x7f850010a9d0 2024-09-10T23:30:11.903 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-09-10T23:30:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.905+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 msgr2=0x7f84e40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.905+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 0x7f84e40b7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f84f0000f80 tx=0x7f84f0031000 comp rx=0 tx=0).stop 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.905+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 msgr2=0x7f850019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.905+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e800d770 tx=0x7f84e800dc30 comp rx=0 tx=0).stop 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 shutdown_connections 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f84e40b54d0 0x7f84e40b7990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f850010b350 0x7f85001a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850010a9d0 0x7f850019b970 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 --2- 172.21.15.67:0/1479045489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8500104fc0 0x7f850019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:11.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 >> 172.21.15.67:0/1479045489 conn(0x7f85001009f0 msgr2=0x7f8500102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:11.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.906+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 shutdown_connections 2024-09-10T23:30:11.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:11.907+0000 7f8505000640 1 -- 172.21.15.67:0/1479045489 wait complete. 2024-09-10T23:30:12.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:11 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:12.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:11 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:12.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:11 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:12.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T23:30:13.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:12 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:12 smithi067 ceph-mon[30939]: osdmap e140: 12 total, 12 up, 12 in 2024-09-10T23:30:13.048 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:12 smithi067 ceph-mon[30939]: pgmap v1378: 33 pgs: 33 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:13.087 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:30:13.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:12 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:13.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:12 smithi092 ceph-mon[34223]: osdmap e140: 12 total, 12 up, 12 in 2024-09-10T23:30:13.271 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:12 smithi092 ceph-mon[34223]: pgmap v1378: 33 pgs: 33 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:13.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:12 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/1479045489' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:13.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:12 smithi145 ceph-mon[34397]: osdmap e140: 12 total, 12 up, 12 in 2024-09-10T23:30:13.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:12 smithi145 ceph-mon[34397]: pgmap v1378: 33 pgs: 33 active+clean; 14 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 -- 172.21.15.67:0/484858231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 msgr2=0x7f8d4c1117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 --2- 172.21.15.67:0/484858231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1117b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009ca0 tx=0x7f8d4802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 -- 172.21.15.67:0/484858231 shutdown_connections 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 --2- 172.21.15.67:0/484858231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1117b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 --2- 172.21.15.67:0/484858231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075f40 0x7f8d4c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 --2- 172.21.15.67:0/484858231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075540 0x7f8d4c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 -- 172.21.15.67:0/484858231 >> 172.21.15.67:0/484858231 conn(0x7f8d4c0fe810 msgr2=0x7f8d4c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 -- 172.21.15.67:0/484858231 shutdown_connections 2024-09-10T23:30:13.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.620+0000 7f8d54800640 1 -- 172.21.15.67:0/484858231 wait complete. 2024-09-10T23:30:13.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.621+0000 7f8d54800640 1 Processor -- start 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 -- start start 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075540 0x7f8d4c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075f40 0x7f8d4c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c114eb0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c114d30 con 0x7f8d4c075f40 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d54800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c115030 con 0x7f8d4c075540 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d52800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:13.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d51e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075540 0x7f8d4c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:13.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d52800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53424/0 (socket says 172.21.15.67:53424) 2024-09-10T23:30:13.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d52800640 1 -- 172.21.15.67:0/639124690 learned_addr learned my addr 172.21.15.67:0/639124690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-09-10T23:30:13.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.622+0000 7f8d51400640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075f40 0x7f8d4c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:13.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 -- 172.21.15.67:0/639124690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075540 msgr2=0x7f8d4c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:13.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075540 0x7f8d4c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 -- 172.21.15.67:0/639124690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075f40 msgr2=0x7f8d4c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:13.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075f40 0x7f8d4c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 -- 172.21.15.67:0/639124690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c1a1530 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.623+0000 7f8d52800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d48031040 tx=0x7f8d48032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.624+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48032eb0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.624+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d480044d0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.624+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48030ca0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.624+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c1a17c0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.624+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1a7ee0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.625+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c075920 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.629+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8d48004040 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.630+0000 7f8d46a00640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 0x7f8d340b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:30:13.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.630+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8539+0+0 (secure 0 0 0) 0x7f8d481074e0 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.630+0000 7f8d51e00640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 0x7f8d340b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:30:13.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.631+0000 7f8d51e00640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 0x7f8d340b7990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000abe0 tx=0x7f8d4000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:30:13.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.632+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d480ce080 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.835+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d4c1a8170 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.918+0000 7f8d46a00640 1 -- 172.21.15.67:0/639124690 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v141) ==== 178+0+0 (secure 0 0 0) 0x7f8d480cd980 con 0x7f8d4c10f3c0 2024-09-10T23:30:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.920+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 msgr2=0x7f8d340b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.920+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 0x7f8d340b7990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000abe0 tx=0x7f8d4000a2d0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.920+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 msgr2=0x7f8d4c1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.920+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d48031040 tx=0x7f8d48032bc0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 shutdown_connections 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:6800/1232465191,v1:172.21.15.67:6801/1232465191] conn(0x7f8d340b54d0 0x7f8d340b7990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d4c10f3c0 0x7f8d4c1a0d60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4c075f40 0x7f8d4c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 --2- 172.21.15.67:0/639124690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d4c075540 0x7f8d4c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:30:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 >> 172.21.15.67:0/639124690 conn(0x7f8d4c0fe810 msgr2=0x7f8d4c1115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:30:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 shutdown_connections 2024-09-10T23:30:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-09-10T23:30:13.921+0000 7f8d54800640 1 -- 172.21.15.67:0/639124690 wait complete. 2024-09-10T23:30:14.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:13 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:14.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:13 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:14.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:13 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:30:14.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:30:14.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:30:15.297 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:14 smithi067 ceph-mon[30939]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:14 smithi067 ceph-mon[30939]: osdmap e141: 12 total, 12 up, 12 in 2024-09-10T23:30:15.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:14 smithi067 ceph-mon[30939]: pgmap v1380: 1 pgs: 1 active+clean; 577 KiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:15.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:14 smithi092 ceph-mon[34223]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:15.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:15 smithi092 ceph-mon[34223]: osdmap e141: 12 total, 12 up, 12 in 2024-09-10T23:30:15.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:15 smithi092 ceph-mon[34223]: pgmap v1380: 1 pgs: 1 active+clean; 577 KiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:15.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:14 smithi145 ceph-mon[34397]: from='client.? 172.21.15.67:0/639124690' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:30:15.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:14 smithi145 ceph-mon[34397]: osdmap e141: 12 total, 12 up, 12 in 2024-09-10T23:30:15.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:14 smithi145 ceph-mon[34397]: pgmap v1380: 1 pgs: 1 active+clean; 577 KiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:30:15.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:30:15.498 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T23:30:15.498 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:30:15.588 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:30:15.623 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:30:15.676 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T23:30:15.676 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T23:30:15.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a 2024-09-10T23:30:16.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:15 smithi067 systemd[1]: Stopping Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:16.728 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a[30935]: 2024-09-10T23:30:16.455+0000 7f5bb6000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:30:16.728 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a[30935]: 2024-09-10T23:30:16.456+0000 7f5bb6000640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T23:30:17.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:16 smithi067 podman[139590]: 2024-09-10 23:30:16.727450153 +0000 UTC m=+0.457967629 container died f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T23:30:17.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:17 smithi067 podman[139590]: 2024-09-10 23:30:17.189037726 +0000 UTC m=+0.919555189 container cleanup f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T23:30:17.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:17 smithi067 bash[139590]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a 2024-09-10T23:30:17.798 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:17 smithi067 podman[139600]: 2024-09-10 23:30:17.474482268 +0000 UTC m=+0.722968313 container remove f3b285da6f5d9622ff4bafba670035448d55c3bac64013f511af93cdfc6ae1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T23:30:18.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T23:30:18.298 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:18 smithi067 systemd[1]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T23:30:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:18 smithi067 systemd[1]: Stopped Ceph mon.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968. 2024-09-10T23:30:18.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Sep 10 23:30:18 smithi067 systemd[1]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.a.service: Consumed 41.301s CPU time. 2024-09-10T23:30:19.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:19.010 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T23:30:19.010 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T23:30:19.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.b 2024-09-10T23:30:19.521 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:19 smithi092 systemd[1]: Stopping Ceph mon.b for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b[34219]: 2024-09-10T23:30:23.678+0000 7f0e84400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:30:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b[34219]: 2024-09-10T23:30:23.678+0000 7f0e84400640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T23:30:24.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:23 smithi092 podman[75780]: 2024-09-10 23:30:23.984495757 +0000 UTC m=+4.310631971 container died 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-10T23:30:25.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:24 smithi092 podman[75780]: 2024-09-10 23:30:24.573381887 +0000 UTC m=+4.899518100 container cleanup 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T23:30:25.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:24 smithi092 bash[75780]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b 2024-09-10T23:30:25.021 INFO:journalctl@ceph.mon.b.smithi092.stdout:Sep 10 23:30:24 smithi092 podman[75791]: 2024-09-10 23:30:24.817886733 +0000 UTC m=+0.810971246 container remove 7740611aba9bcda6973aeb7e068620043ebce061eb68471873e263433346d260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T23:30:25.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T23:30:25.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:25.464 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T23:30:25.464 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T23:30:25.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.c 2024-09-10T23:30:25.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:25 smithi145 systemd[1]: Stopping Ceph mon.c for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:29.067 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:28 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c[34393]: 2024-09-10T23:30:28.807+0000 7f43e0600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:30:29.067 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:28 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c[34393]: 2024-09-10T23:30:28.807+0000 7f43e0600640 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-09-10T23:30:29.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:29 smithi145 podman[78228]: 2024-09-10 23:30:29.06629973 +0000 UTC m=+0.668142685 container died 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T23:30:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:35 smithi145 podman[78228]: 2024-09-10 23:30:35.823855563 +0000 UTC m=+7.425698516 container cleanup 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T23:30:36.130 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:35 smithi145 bash[78228]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c 2024-09-10T23:30:36.505 INFO:journalctl@ceph.mon.c.smithi145.stdout:Sep 10 23:30:36 smithi145 podman[78240]: 2024-09-10 23:30:36.209251682 +0000 UTC m=+7.133383467 container remove 325a515fb55553e47097ada51d1e949019b69207ccc11bae42632573114de992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T23:30:36.534 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T23:30:36.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:36.685 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T23:30:36.685 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T23:30:36.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x 2024-09-10T23:30:37.048 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:36 smithi067 systemd[1]: Stopping Ceph mgr.x for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:44.048 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:43 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x[31175]: 2024-09-10T23:30:43.733+0000 7fcaad400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:30:45.047 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:44 smithi067 podman[139706]: 2024-09-10 23:30:44.773973572 +0000 UTC m=+5.596699185 container died b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:30:45.899 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:45 smithi067 podman[139706]: 2024-09-10 23:30:45.572395195 +0000 UTC m=+6.395120807 container cleanup b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T23:30:45.900 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:45 smithi067 bash[139706]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x 2024-09-10T23:30:46.160 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 10 23:30:45 smithi067 podman[139720]: 2024-09-10 23:30:45.899054308 +0000 UTC m=+1.073425598 container remove b9ff716034b7961464745ed0f76e29281ad5fd6f401a4c4f0ab27bb528251734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:30:46.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T23:30:46.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:46.245 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T23:30:46.245 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T23:30:46.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.y 2024-09-10T23:30:46.771 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:46 smithi092 systemd[1]: Stopping Ceph mgr.y for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:47.270 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:46 smithi092 podman[75908]: 2024-09-10 23:30:46.804549949 +0000 UTC m=+0.214126733 container died a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T23:30:50.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:50 smithi092 podman[75908]: 2024-09-10 23:30:50.294067595 +0000 UTC m=+3.703644377 container cleanup a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T23:30:50.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:50 smithi092 bash[75908]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y 2024-09-10T23:30:50.829 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:50 smithi092 podman[75919]: 2024-09-10 23:30:50.570878524 +0000 UTC m=+3.749796921 container remove a10462e9fe10ff09eb651fc50f47b3aecb682b6f02434202f9f056ab45fefa0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-10T23:30:50.829 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 10 23:30:50 smithi092 systemd[1]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T23:30:50.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T23:30:50.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:50.913 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T23:30:50.913 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T23:30:50.914 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.z 2024-09-10T23:30:51.380 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 23:30:51 smithi145 systemd[1]: Stopping Ceph mgr.z for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:51.821 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 23:30:51 smithi145 podman[78357]: 2024-09-10 23:30:51.468783935 +0000 UTC m=+0.243917905 container died 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T23:30:52.129 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 23:30:51 smithi145 podman[78357]: 2024-09-10 23:30:51.945353784 +0000 UTC m=+0.720487745 container cleanup 2ea72b5cedbb6242f43fdb620e8f884127c8f3c8f619a40136d48d832be5edf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T23:30:52.130 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Sep 10 23:30:51 smithi145 bash[78357]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mgr-z 2024-09-10T23:30:52.482 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T23:30:52.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:30:52.557 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T23:30:52.557 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T23:30:52.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.0 2024-09-10T23:30:52.990 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:52 smithi067 systemd[1]: Stopping Ceph osd.0 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:30:53.298 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:52 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T23:30:52.988+0000 7f154cc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:30:53.298 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:52 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T23:30:52.988+0000 7f154cc00640 -1 osd.0 141 *** Got signal Terminated *** 2024-09-10T23:30:53.298 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:52 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0[48607]: 2024-09-10T23:30:52.988+0000 7f154cc00640 -1 osd.0 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:30:59.205 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:58 smithi067 podman[139838]: 2024-09-10 23:30:58.880106502 +0000 UTC m=+6.023079539 container died d75d82ba7a1420cb3ca0f288ba837dedc14d23618aa29bc655159e60e595802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T23:30:59.547 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:59 smithi067 podman[139838]: 2024-09-10 23:30:59.420106584 +0000 UTC m=+6.563079619 container cleanup d75d82ba7a1420cb3ca0f288ba837dedc14d23618aa29bc655159e60e595802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:30:59.548 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:59 smithi067 bash[139838]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0 2024-09-10T23:31:00.047 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:30:59 smithi067 podman[139856]: 2024-09-10 23:30:59.638726709 +0000 UTC m=+0.750128209 container remove d75d82ba7a1420cb3ca0f288ba837dedc14d23618aa29bc655159e60e595802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:31:00.492 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:00 smithi067 podman[139943]: 2024-09-10 23:31:00.093071915 +0000 UTC m=+0.027477908 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:00.492 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:00 smithi067 podman[139943]: 2024-09-10 23:31:00.231320094 +0000 UTC m=+0.165726083 container create 993a45eb8bf3164a2f6d7279ac12019b4b4de28d4357e740dd14fb78a3e935b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T23:31:00.797 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:00 smithi067 podman[139943]: 2024-09-10 23:31:00.492485625 +0000 UTC m=+0.426891618 container init 993a45eb8bf3164a2f6d7279ac12019b4b4de28d4357e740dd14fb78a3e935b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T23:31:00.798 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:00 smithi067 podman[139943]: 2024-09-10 23:31:00.499466932 +0000 UTC m=+0.433872926 container start 993a45eb8bf3164a2f6d7279ac12019b4b4de28d4357e740dd14fb78a3e935b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:31:00.798 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:00 smithi067 podman[139943]: 2024-09-10 23:31:00.592489065 +0000 UTC m=+0.526895062 container attach 993a45eb8bf3164a2f6d7279ac12019b4b4de28d4357e740dd14fb78a3e935b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T23:31:01.413 INFO:journalctl@ceph.osd.0.smithi067.stdout:Sep 10 23:31:01 smithi067 podman[139943]: 2024-09-10 23:31:01.094769781 +0000 UTC m=+1.029175775 container died 993a45eb8bf3164a2f6d7279ac12019b4b4de28d4357e740dd14fb78a3e935b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-10T23:31:01.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T23:31:01.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:01.971 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T23:31:01.971 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T23:31:01.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.1 2024-09-10T23:31:02.410 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:02 smithi092 systemd[1]: Stopping Ceph osd.1 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:02.771 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:02 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1[39966]: 2024-09-10T23:31:02.408+0000 7f0f64e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:02.771 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:02 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1[39966]: 2024-09-10T23:31:02.408+0000 7f0f64e00640 -1 osd.1 141 *** Got signal Terminated *** 2024-09-10T23:31:02.771 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:02 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1[39966]: 2024-09-10T23:31:02.408+0000 7f0f64e00640 -1 osd.1 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:08.591 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:08 smithi092 podman[76032]: 2024-09-10 23:31:08.270744658 +0000 UTC m=+5.996039114 container died 8042b7d66d11c28d48ef93192f3489afbedc2f3b6841e87dbfdb356750873474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:31:08.966 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:08 smithi092 podman[76032]: 2024-09-10 23:31:08.722161923 +0000 UTC m=+6.447456388 container cleanup 8042b7d66d11c28d48ef93192f3489afbedc2f3b6841e87dbfdb356750873474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T23:31:08.966 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:08 smithi092 bash[76032]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1 2024-09-10T23:31:09.270 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:08 smithi092 podman[76048]: 2024-09-10 23:31:08.965293557 +0000 UTC m=+0.684897238 container remove 8042b7d66d11c28d48ef93192f3489afbedc2f3b6841e87dbfdb356750873474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:31:09.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:09 smithi092 podman[76135]: 2024-09-10 23:31:09.438159178 +0000 UTC m=+0.026605898 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:09.861 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:09 smithi092 podman[76135]: 2024-09-10 23:31:09.549683108 +0000 UTC m=+0.138129829 container create a1bd9a1cdf7320a963592eed5198f8b7caa99662aed58932ff245318456f1c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:31:10.271 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:09 smithi092 podman[76135]: 2024-09-10 23:31:09.861268059 +0000 UTC m=+0.449714780 container init a1bd9a1cdf7320a963592eed5198f8b7caa99662aed58932ff245318456f1c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T23:31:10.271 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:09 smithi092 podman[76135]: 2024-09-10 23:31:09.868312144 +0000 UTC m=+0.456758865 container start a1bd9a1cdf7320a963592eed5198f8b7caa99662aed58932ff245318456f1c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T23:31:10.271 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:09 smithi092 podman[76135]: 2024-09-10 23:31:09.994414631 +0000 UTC m=+0.582861361 container attach a1bd9a1cdf7320a963592eed5198f8b7caa99662aed58932ff245318456f1c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T23:31:10.665 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 10 23:31:10 smithi092 podman[76135]: 2024-09-10 23:31:10.33255211 +0000 UTC m=+0.920998840 container died a1bd9a1cdf7320a963592eed5198f8b7caa99662aed58932ff245318456f1c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:31:11.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T23:31:11.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:11.201 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T23:31:11.201 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T23:31:11.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.2 2024-09-10T23:31:11.515 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:11 smithi145 systemd[1]: Stopping Ceph osd.2 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:11.857 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T23:31:11.652+0000 7f94b9200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:11.858 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T23:31:11.652+0000 7f94b9200640 -1 osd.2 141 *** Got signal Terminated *** 2024-09-10T23:31:11.858 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2[40551]: 2024-09-10T23:31:11.652+0000 7f94b9200640 -1 osd.2 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:18.035 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:17 smithi145 podman[78482]: 2024-09-10 23:31:17.659276939 +0000 UTC m=+6.166489228 container stop ce21eb539f75d63a3dba2f8612898f4687fd6f4bf501a5b2aeb43bc51e4b10d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T23:31:18.035 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:17 smithi145 podman[78482]: 2024-09-10 23:31:17.68670724 +0000 UTC m=+6.193919526 container died ce21eb539f75d63a3dba2f8612898f4687fd6f4bf501a5b2aeb43bc51e4b10d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:31:18.602 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:18 smithi145 podman[78482]: 2024-09-10 23:31:18.307928694 +0000 UTC m=+6.815140979 container cleanup ce21eb539f75d63a3dba2f8612898f4687fd6f4bf501a5b2aeb43bc51e4b10d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T23:31:18.602 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:18 smithi145 bash[78482]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2 2024-09-10T23:31:18.879 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:18 smithi145 podman[78511]: 2024-09-10 23:31:18.601670601 +0000 UTC m=+0.929273869 container remove ce21eb539f75d63a3dba2f8612898f4687fd6f4bf501a5b2aeb43bc51e4b10d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T23:31:19.464 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:19 smithi145 podman[78601]: 2024-09-10 23:31:19.050648658 +0000 UTC m=+0.027366499 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:19.464 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:19 smithi145 podman[78601]: 2024-09-10 23:31:19.186155834 +0000 UTC m=+0.162873657 container create 9e22e243a5910faca652e0dcf99d32bada993886a846612ec084d34e3f052fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:31:19.880 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:19 smithi145 podman[78601]: 2024-09-10 23:31:19.46383851 +0000 UTC m=+0.440556362 container init 9e22e243a5910faca652e0dcf99d32bada993886a846612ec084d34e3f052fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:31:19.880 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:19 smithi145 podman[78601]: 2024-09-10 23:31:19.470684468 +0000 UTC m=+0.447402292 container start 9e22e243a5910faca652e0dcf99d32bada993886a846612ec084d34e3f052fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T23:31:19.880 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:19 smithi145 podman[78601]: 2024-09-10 23:31:19.571927156 +0000 UTC m=+0.548644980 container attach 9e22e243a5910faca652e0dcf99d32bada993886a846612ec084d34e3f052fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T23:31:20.359 INFO:journalctl@ceph.osd.2.smithi145.stdout:Sep 10 23:31:20 smithi145 podman[78624]: 2024-09-10 23:31:20.001725403 +0000 UTC m=+0.026417489 container died 9e22e243a5910faca652e0dcf99d32bada993886a846612ec084d34e3f052fd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:31:20.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T23:31:20.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:20.834 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T23:31:20.834 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T23:31:20.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.3 2024-09-10T23:31:21.219 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:20 smithi067 systemd[1]: Stopping Ceph osd.3 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:21.547 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T23:31:21.268+0000 7fda4f600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:21.548 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T23:31:21.268+0000 7fda4f600640 -1 osd.3 141 *** Got signal Terminated *** 2024-09-10T23:31:21.548 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3[55344]: 2024-09-10T23:31:21.268+0000 7fda4f600640 -1 osd.3 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:27.777 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:27 smithi067 podman[140044]: 2024-09-10 23:31:27.472482007 +0000 UTC m=+6.342064599 container died effedfff442947e15953532d20d1d54511fdbd31462514f5cc427fd75123d265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:31:28.037 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:28 smithi067 podman[140044]: 2024-09-10 23:31:28.016893371 +0000 UTC m=+6.886475967 container cleanup effedfff442947e15953532d20d1d54511fdbd31462514f5cc427fd75123d265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:31:28.037 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:28 smithi067 bash[140044]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3 2024-09-10T23:31:28.797 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:28 smithi067 podman[140074]: 2024-09-10 23:31:28.39373076 +0000 UTC m=+0.912031603 container remove effedfff442947e15953532d20d1d54511fdbd31462514f5cc427fd75123d265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T23:31:29.223 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:28 smithi067 podman[140163]: 2024-09-10 23:31:28.846951717 +0000 UTC m=+0.027565260 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:29.223 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:28 smithi067 podman[140163]: 2024-09-10 23:31:28.961371511 +0000 UTC m=+0.141985034 container create 0f4952e61ab2939469d45c6c8fb3bbe5f458dac121b9401f70fe35be05618e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T23:31:29.223 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:29 smithi067 podman[140163]: 2024-09-10 23:31:29.222497285 +0000 UTC m=+0.403110834 container init 0f4952e61ab2939469d45c6c8fb3bbe5f458dac121b9401f70fe35be05618e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:31:29.498 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:29 smithi067 podman[140163]: 2024-09-10 23:31:29.230084517 +0000 UTC m=+0.410698040 container start 0f4952e61ab2939469d45c6c8fb3bbe5f458dac121b9401f70fe35be05618e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:31:29.498 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:29 smithi067 podman[140163]: 2024-09-10 23:31:29.347632763 +0000 UTC m=+0.528246287 container attach 0f4952e61ab2939469d45c6c8fb3bbe5f458dac121b9401f70fe35be05618e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T23:31:29.797 INFO:journalctl@ceph.osd.3.smithi067.stdout:Sep 10 23:31:29 smithi067 podman[140163]: 2024-09-10 23:31:29.498504345 +0000 UTC m=+0.679117868 container died 0f4952e61ab2939469d45c6c8fb3bbe5f458dac121b9401f70fe35be05618e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T23:31:30.254 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T23:31:30.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:30.332 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T23:31:30.333 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T23:31:30.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.4 2024-09-10T23:31:30.728 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:30 smithi092 systemd[1]: Stopping Ceph osd.4 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:31.020 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T23:31:30.726+0000 7fa3be600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:31.021 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T23:31:30.726+0000 7fa3be600640 -1 osd.4 141 *** Got signal Terminated *** 2024-09-10T23:31:31.021 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4[46579]: 2024-09-10T23:31:30.726+0000 7fa3be600640 -1 osd.4 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:37.191 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:36 smithi092 podman[76250]: 2024-09-10 23:31:36.893185445 +0000 UTC m=+6.281812257 container died ae042f590cf58678c0a6d3cea36a9c968cc90b362f17549f1cd57da90409952e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:31:37.521 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:37 smithi092 podman[76250]: 2024-09-10 23:31:37.30676345 +0000 UTC m=+6.695390261 container cleanup ae042f590cf58678c0a6d3cea36a9c968cc90b362f17549f1cd57da90409952e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T23:31:37.521 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:37 smithi092 bash[76250]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4 2024-09-10T23:31:38.021 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:37 smithi092 podman[76269]: 2024-09-10 23:31:37.616604335 +0000 UTC m=+0.710985642 container remove ae042f590cf58678c0a6d3cea36a9c968cc90b362f17549f1cd57da90409952e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:31:38.383 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:37.968039936 +0000 UTC m=+0.039525393 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:38.383 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:38.109065093 +0000 UTC m=+0.180550546 container create 3edcd2875dcdbb79e47b6caa1357677fff2aece34ebd4d31c37f0c61b9f6607a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:31:38.384 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:38.383428523 +0000 UTC m=+0.454913973 container init 3edcd2875dcdbb79e47b6caa1357677fff2aece34ebd4d31c37f0c61b9f6607a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-10T23:31:38.652 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:38.390236241 +0000 UTC m=+0.461721702 container start 3edcd2875dcdbb79e47b6caa1357677fff2aece34ebd4d31c37f0c61b9f6607a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T23:31:38.652 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:38.536907517 +0000 UTC m=+0.608393004 container attach 3edcd2875dcdbb79e47b6caa1357677fff2aece34ebd4d31c37f0c61b9f6607a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T23:31:39.006 INFO:journalctl@ceph.osd.4.smithi092.stdout:Sep 10 23:31:38 smithi092 podman[76355]: 2024-09-10 23:31:38.652243294 +0000 UTC m=+0.723728760 container died 3edcd2875dcdbb79e47b6caa1357677fff2aece34ebd4d31c37f0c61b9f6607a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T23:31:39.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T23:31:39.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:39.506 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T23:31:39.506 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T23:31:39.506 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.5 2024-09-10T23:31:39.879 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:39 smithi145 systemd[1]: Stopping Ceph osd.5 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:40.380 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T23:31:39.955+0000 7f1761200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:40.380 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T23:31:39.955+0000 7f1761200640 -1 osd.5 141 *** Got signal Terminated *** 2024-09-10T23:31:40.380 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5[47114]: 2024-09-10T23:31:39.955+0000 7f1761200640 -1 osd.5 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:46.529 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:46 smithi145 podman[78702]: 2024-09-10 23:31:46.165985537 +0000 UTC m=+6.363543782 container died 7a96a474386a640b3ed33f3467d22dd9786c5b837e46192cefd92403f0df1dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T23:31:46.879 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:46 smithi145 podman[78702]: 2024-09-10 23:31:46.660983192 +0000 UTC m=+6.858541436 container cleanup 7a96a474386a640b3ed33f3467d22dd9786c5b837e46192cefd92403f0df1dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T23:31:46.880 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:46 smithi145 bash[78702]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5 2024-09-10T23:31:47.379 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:46 smithi145 podman[78720]: 2024-09-10 23:31:46.960542849 +0000 UTC m=+0.781508176 container remove 7a96a474386a640b3ed33f3467d22dd9786c5b837e46192cefd92403f0df1dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:31:47.782 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:47 smithi145 podman[78801]: 2024-09-10 23:31:47.513561301 +0000 UTC m=+0.110222178 container create 475c7cd1fb0976db20ebd1dc4b86a904331582d949d188c23a0f9a8ff9ecca9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-10T23:31:47.783 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:47 smithi145 podman[78801]: 2024-09-10 23:31:47.430330949 +0000 UTC m=+0.026991834 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:47.783 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:47 smithi145 podman[78801]: 2024-09-10 23:31:47.782498286 +0000 UTC m=+0.379159184 container init 475c7cd1fb0976db20ebd1dc4b86a904331582d949d188c23a0f9a8ff9ecca9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T23:31:48.049 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:47 smithi145 podman[78801]: 2024-09-10 23:31:47.789344153 +0000 UTC m=+0.386005030 container start 475c7cd1fb0976db20ebd1dc4b86a904331582d949d188c23a0f9a8ff9ecca9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:31:48.049 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:47 smithi145 podman[78801]: 2024-09-10 23:31:47.891006891 +0000 UTC m=+0.487667771 container attach 475c7cd1fb0976db20ebd1dc4b86a904331582d949d188c23a0f9a8ff9ecca9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T23:31:48.380 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 10 23:31:48 smithi145 podman[78801]: 2024-09-10 23:31:48.04905116 +0000 UTC m=+0.645712039 container died 475c7cd1fb0976db20ebd1dc4b86a904331582d949d188c23a0f9a8ff9ecca9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-10T23:31:48.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T23:31:48.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:48.925 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T23:31:48.927 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T23:31:48.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.6 2024-09-10T23:31:49.297 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:49 smithi067 systemd[1]: Stopping Ceph osd.6 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:49.798 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:49 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T23:31:49.365+0000 7f2a81c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:49.798 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:49 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T23:31:49.365+0000 7f2a81c00640 -1 osd.6 141 *** Got signal Terminated *** 2024-09-10T23:31:49.798 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:49 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6[62257]: 2024-09-10T23:31:49.365+0000 7f2a81c00640 -1 osd.6 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:31:55.739 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:55 smithi067 podman[140262]: 2024-09-10 23:31:55.428238219 +0000 UTC m=+6.207138803 container died 8129aa5e7244cfde0b6fad477a95f4b4ebfbb9a39c7575329ed5b343366dae15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:31:55.999 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:55 smithi067 podman[140262]: 2024-09-10 23:31:55.97982042 +0000 UTC m=+6.758721004 container cleanup 8129aa5e7244cfde0b6fad477a95f4b4ebfbb9a39c7575329ed5b343366dae15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:31:55.999 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:55 smithi067 bash[140262]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6 2024-09-10T23:31:56.547 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:56 smithi067 podman[140279]: 2024-09-10 23:31:56.189846354 +0000 UTC m=+0.751614111 container remove 8129aa5e7244cfde0b6fad477a95f4b4ebfbb9a39c7575329ed5b343366dae15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T23:31:57.018 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:56 smithi067 podman[140368]: 2024-09-10 23:31:56.652873985 +0000 UTC m=+0.027402072 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:31:57.018 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:56 smithi067 podman[140368]: 2024-09-10 23:31:56.790823109 +0000 UTC m=+0.165351175 container create 2b5ea73c44fc57d341efde17c66cc16276c7ef98f215fee30b85f2a53d9d3024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T23:31:57.294 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:57 smithi067 podman[140368]: 2024-09-10 23:31:57.018312152 +0000 UTC m=+0.392840239 container init 2b5ea73c44fc57d341efde17c66cc16276c7ef98f215fee30b85f2a53d9d3024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:31:57.295 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:57 smithi067 podman[140368]: 2024-09-10 23:31:57.025394274 +0000 UTC m=+0.399922339 container start 2b5ea73c44fc57d341efde17c66cc16276c7ef98f215fee30b85f2a53d9d3024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T23:31:57.295 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:57 smithi067 podman[140368]: 2024-09-10 23:31:57.134868253 +0000 UTC m=+0.509396325 container attach 2b5ea73c44fc57d341efde17c66cc16276c7ef98f215fee30b85f2a53d9d3024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T23:31:57.547 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 10 23:31:57 smithi067 podman[140368]: 2024-09-10 23:31:57.294420742 +0000 UTC m=+0.668948814 container died 2b5ea73c44fc57d341efde17c66cc16276c7ef98f215fee30b85f2a53d9d3024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:31:57.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T23:31:58.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:31:58.047 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T23:31:58.047 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T23:31:58.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.7 2024-09-10T23:31:58.437 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:31:58 smithi092 systemd[1]: Stopping Ceph osd.7 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:31:58.770 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:31:58 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T23:31:58.436+0000 7f3c0ae00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:31:58.771 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:31:58 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T23:31:58.436+0000 7f3c0ae00640 -1 osd.7 141 *** Got signal Terminated *** 2024-09-10T23:31:58.771 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:31:58 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7[53201]: 2024-09-10T23:31:58.436+0000 7f3c0ae00640 -1 osd.7 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:32:00.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:31:59 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:31:59.819+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:00.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:00 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:00.115+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:01.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:00 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:00.866+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:01.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:01 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:01.088+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:01.380 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:01 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:01.160+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:02.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:01 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:01.848+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:02.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:02 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:02.117+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:02.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:02 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:02.102+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:03.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:02 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:02.807+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:03.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:03 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:03.109+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:03.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:03 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:03.085+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:04.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:03 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:03.842+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:04.380 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:04 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:04.087+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:04.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:04 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:04.099+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:04.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:04 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:04.353+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:04.971 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:04 smithi092 podman[76470]: 2024-09-10 23:32:04.665611336 +0000 UTC m=+6.340669467 container died f9f8e9bc3b6db901bc8cad7fd869cf5bddab0faeb23d45547c4c54e8049639dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:32:04.971 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:04 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:04.823+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:05.271 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:05 smithi092 podman[76470]: 2024-09-10 23:32:05.115611964 +0000 UTC m=+6.790670110 container cleanup f9f8e9bc3b6db901bc8cad7fd869cf5bddab0faeb23d45547c4c54e8049639dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T23:32:05.271 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:05 smithi092 bash[76470]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7 2024-09-10T23:32:05.380 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:05.102+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:05.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:05 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:05.065+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:05.770 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:05 smithi092 podman[76485]: 2024-09-10 23:32:05.300171832 +0000 UTC m=+0.623244203 container remove f9f8e9bc3b6db901bc8cad7fd869cf5bddab0faeb23d45547c4c54e8049639dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:32:05.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:05 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:05.338+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:06.195 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:05 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:05.819+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:06.196 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:05 smithi092 podman[76569]: 2024-09-10 23:32:05.773606015 +0000 UTC m=+0.038041284 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:32:06.196 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:05 smithi092 podman[76569]: 2024-09-10 23:32:05.901000719 +0000 UTC m=+0.165435978 container create 2f898bce5a9e7bd799335cc7ed5a06a1f474ce6f9a50362f031898313bf65a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-10T23:32:06.196 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:06 smithi092 podman[76569]: 2024-09-10 23:32:06.195299796 +0000 UTC m=+0.459735077 container init 2f898bce5a9e7bd799335cc7ed5a06a1f474ce6f9a50362f031898313bf65a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T23:32:06.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:06 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:06.060+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:06.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:06 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:06.069+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:06.464 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:06 smithi092 podman[76569]: 2024-09-10 23:32:06.202212718 +0000 UTC m=+0.466647979 container start 2f898bce5a9e7bd799335cc7ed5a06a1f474ce6f9a50362f031898313bf65a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:32:06.465 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:06 smithi092 podman[76569]: 2024-09-10 23:32:06.303189753 +0000 UTC m=+0.567625015 container attach 2f898bce5a9e7bd799335cc7ed5a06a1f474ce6f9a50362f031898313bf65a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T23:32:06.770 INFO:journalctl@ceph.osd.7.smithi092.stdout:Sep 10 23:32:06 smithi092 podman[76569]: 2024-09-10 23:32:06.464906551 +0000 UTC m=+0.729341823 container died 2f898bce5a9e7bd799335cc7ed5a06a1f474ce6f9a50362f031898313bf65a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T23:32:06.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:06 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:06.311+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:07.132 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:06 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:06.819+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:07.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T23:32:07.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:07.242 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T23:32:07.242 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T23:32:07.242 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.8 2024-09-10T23:32:07.296 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:07 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:07.039+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:07.296 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:07 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:07.067+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:07.629 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:07 smithi145 systemd[1]: Stopping Ceph osd.8 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:07.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:07 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:07.336+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:08.069 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:07 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:07.708+0000 7fe017c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:08.069 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:07 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:07.708+0000 7fe017c00640 -1 osd.8 141 *** Got signal Terminated *** 2024-09-10T23:32:08.069 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:07 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:07.708+0000 7fe017c00640 -1 osd.8 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:32:08.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:07 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:07.836+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:08.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:08 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:08.068+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:08.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:08 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:08.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:08.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:08 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:08.367+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:08.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:08 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:08.367+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:09.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:08 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:08.859+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:09.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:09 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:09.100+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:09.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:09 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:09.091+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:09.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:09 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:09.373+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:09.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:09 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:09.373+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:10.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:09 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:09.844+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:10.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:10 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:10.082+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:10.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:10 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:10.074+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:10.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:10 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:10.358+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:10.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:10 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:10.358+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:11.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:10 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:10.847+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:11.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:11.071+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:11.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:11 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:11.092+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:11.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:11 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:11.337+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:11.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:11 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:11.337+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:12.172 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:11.847+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:12.379 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:12 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8[54129]: 2024-09-10T23:32:12.038+0000 7fe00e600640 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:55.527958+0000 front 2024-09-10T23:30:55.527834+0000 (oldest deadline 2024-09-10T23:32:00.827931+0000) 2024-09-10T23:32:12.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:12 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:12.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:12.797 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:12 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:12.308+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:12.798 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:12 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:12.308+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:13.192 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:12 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:12.863+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:13.192 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:12 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:12.863+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:13.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:13 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:13.065+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:13.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:13 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:13.065+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:13.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:13 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:13.276+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:13.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:13 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:13.276+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:13.823 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:13 smithi145 podman[78916]: 2024-09-10 23:32:13.45949589 +0000 UTC m=+5.930531402 container stop bca5441bd4469570a69389b403f501fb6ee585803f74d37d417b0da0a333d685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T23:32:13.823 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:13 smithi145 podman[78916]: 2024-09-10 23:32:13.485271155 +0000 UTC m=+5.956306668 container died bca5441bd4469570a69389b403f501fb6ee585803f74d37d417b0da0a333d685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:14.129 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:14 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:14.046+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:14.130 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:14 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:14.046+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:14.130 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:13 smithi145 podman[78916]: 2024-09-10 23:32:13.962734669 +0000 UTC m=+6.433770186 container cleanup bca5441bd4469570a69389b403f501fb6ee585803f74d37d417b0da0a333d685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T23:32:14.130 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:13 smithi145 bash[78916]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8 2024-09-10T23:32:14.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:13 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:13.849+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:14.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:13 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:13.849+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:14.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:14 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:14.248+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:14.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:14 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:14.248+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:14.629 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:14 smithi145 podman[78932]: 2024-09-10 23:32:14.19438656 +0000 UTC m=+0.722142326 container remove bca5441bd4469570a69389b403f501fb6ee585803f74d37d417b0da0a333d685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T23:32:15.008 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:14 smithi145 podman[79018]: 2024-09-10 23:32:14.658219601 +0000 UTC m=+0.027287005 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:32:15.008 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:14 smithi145 podman[79018]: 2024-09-10 23:32:14.781378856 +0000 UTC m=+0.150446266 container create 4117d3d1d464e499f8d9d70a08650040988daa479f7caf394da44afb9a347cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:15.008 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:15 smithi145 podman[79018]: 2024-09-10 23:32:15.001059047 +0000 UTC m=+0.370126451 container init 4117d3d1d464e499f8d9d70a08650040988daa479f7caf394da44afb9a347cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T23:32:15.009 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:15 smithi145 podman[79018]: 2024-09-10 23:32:15.007623317 +0000 UTC m=+0.376690722 container start 4117d3d1d464e499f8d9d70a08650040988daa479f7caf394da44afb9a347cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T23:32:15.020 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:14 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:14.844+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:15.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:14 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:14.844+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:15.266 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:15 smithi145 podman[79018]: 2024-09-10 23:32:15.15889668 +0000 UTC m=+0.527964097 container attach 4117d3d1d464e499f8d9d70a08650040988daa479f7caf394da44afb9a347cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T23:32:15.267 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:15 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:15.023+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:15.267 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:15 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:15.023+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:15.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:15 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:15.266+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:15.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:15 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:15.266+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:15.629 INFO:journalctl@ceph.osd.8.smithi145.stdout:Sep 10 23:32:15 smithi145 podman[79018]: 2024-09-10 23:32:15.266967694 +0000 UTC m=+0.636035101 container died 4117d3d1d464e499f8d9d70a08650040988daa479f7caf394da44afb9a347cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:32:16.129 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:16 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:16.008+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:16.130 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:16 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:16.008+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:16.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T23:32:16.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:15 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:15.824+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:16.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:15 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:15.824+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:16.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:16.318 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T23:32:16.318 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T23:32:16.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.9 2024-09-10T23:32:16.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:16.272+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:16.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:16.272+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:16.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 systemd[1]: Stopping Ceph osd.9 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:17.047 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:16.735+0000 7ff2c2800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:17.048 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:16.735+0000 7ff2c2800640 -1 osd.9 141 *** Got signal Terminated *** 2024-09-10T23:32:17.048 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:16 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:16.735+0000 7ff2c2800640 -1 osd.9 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:32:17.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:16 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:16.796+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:17.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:16 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:16.796+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:17.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:17 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:17.031+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:17.338 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:17 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:17.031+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:17.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:17.248+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:17.879 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:17 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:17.248+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:18.020 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:17 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:17.763+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:18.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:17 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:17.763+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:18.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:18 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:18.057+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:18.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:18 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:18.057+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:18.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:18.217+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:18.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:18 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:18.217+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:19.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:18 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:18.800+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:19.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:18 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:18.800+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:19.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:19 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:19.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:19.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:19 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:19.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:19.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:19 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:19.262+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:19.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:19 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:19.262+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:20.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:19 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:19.805+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:20.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:19 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:19.805+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:20.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:20 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:20.223+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:20.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:20 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:20.223+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:20.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:20 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:20.223+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:15.271126+0000 front 2024-09-10T23:31:15.271009+0000 (oldest deadline 2024-09-10T23:32:19.370843+0000) 2024-09-10T23:32:20.629 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:20 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:20.130+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:20.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:20 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:20.130+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:21.020 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:20 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:20.759+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:21.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:20 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:20.759+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:21.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:20 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:20.759+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:21.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:21 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:21.099+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:21.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:21 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:21.099+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:21.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:21 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:21.099+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:21.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:21.250+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.667082+0000 front 2024-09-10T23:30:53.667051+0000 (oldest deadline 2024-09-10T23:32:03.667816+0000) 2024-09-10T23:32:21.547 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:21.250+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.569232+0000 front 2024-09-10T23:31:06.569223+0000 (oldest deadline 2024-09-10T23:32:08.269312+0000) 2024-09-10T23:32:21.548 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:21 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9[68795]: 2024-09-10T23:32:21.250+0000 7ff2be200640 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:15.271126+0000 front 2024-09-10T23:31:15.271009+0000 (oldest deadline 2024-09-10T23:32:19.370843+0000) 2024-09-10T23:32:22.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:21 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:21.761+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:22.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:21 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:21.761+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:22.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:21 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:21.761+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:22.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:22 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:22.102+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:22.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:22 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:22.102+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:22.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:22 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:22.102+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:23.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:22 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:22.764+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:23.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:22 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:22.764+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:23.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:22 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:22.764+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:23.352 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:23 smithi067 podman[140481]: 2024-09-10 23:32:23.068628968 +0000 UTC m=+6.457401471 container died bc600a645283e85d188fe96cfb48c2a81c5c12e198220d932eea9d0ceeb8473b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:23.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:23 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:23.109+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:23.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:23 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:23.109+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:23.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:23 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:23.109+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:23.726 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:23 smithi067 podman[140481]: 2024-09-10 23:32:23.474320483 +0000 UTC m=+6.863092986 container cleanup bc600a645283e85d188fe96cfb48c2a81c5c12e198220d932eea9d0ceeb8473b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T23:32:23.726 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:23 smithi067 bash[140481]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9 2024-09-10T23:32:24.047 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:23 smithi067 podman[140499]: 2024-09-10 23:32:23.725696021 +0000 UTC m=+0.645779330 container remove bc600a645283e85d188fe96cfb48c2a81c5c12e198220d932eea9d0ceeb8473b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T23:32:24.118 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:23.793+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:24.118 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:23.793+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:24.118 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:23 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:23.793+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:24.446 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.09150267 +0000 UTC m=+0.027440413 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:32:24.446 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.193066563 +0000 UTC m=+0.129004287 container create 1561fa58a29883aefc22671b4cddea2ccfccb0152ddd6f573cfdff31b19eab81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T23:32:24.446 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.445985627 +0000 UTC m=+0.381923375 container init 1561fa58a29883aefc22671b4cddea2ccfccb0152ddd6f573cfdff31b19eab81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:32:24.629 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:24 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:24.143+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:24.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:24 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:24.143+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:24.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:24 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:24.143+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:24.722 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.45325346 +0000 UTC m=+0.389191212 container start 1561fa58a29883aefc22671b4cddea2ccfccb0152ddd6f573cfdff31b19eab81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:32:24.722 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.528793354 +0000 UTC m=+0.464731088 container attach 1561fa58a29883aefc22671b4cddea2ccfccb0152ddd6f573cfdff31b19eab81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T23:32:25.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:24 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:24.748+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:25.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:24 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:24.748+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:25.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:24 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:24.748+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:25.047 INFO:journalctl@ceph.osd.9.smithi067.stdout:Sep 10 23:32:24 smithi067 podman[140586]: 2024-09-10 23:32:24.722189989 +0000 UTC m=+0.658127729 container died 1561fa58a29883aefc22671b4cddea2ccfccb0152ddd6f573cfdff31b19eab81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T23:32:25.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T23:32:25.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:25 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:25.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:25.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:25 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:25.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:25.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:25 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:25.098+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:25.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:25.453 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T23:32:25.453 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T23:32:25.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.10 2024-09-10T23:32:25.770 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 systemd[1]: Stopping Ceph osd.10 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.774+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.774+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.774+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.877+0000 7f47a1800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.877+0000 7f47a1800640 -1 osd.10 141 *** Got signal Terminated *** 2024-09-10T23:32:26.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:25 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:25.877+0000 7f47a1800640 -1 osd.10 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:32:26.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:26 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:26.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:26.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:26 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:26.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:26.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:26 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:26.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:27.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:26 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:26.803+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:27.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:26 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:26.803+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:27.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:26 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:26.803+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:27.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:27.058+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:27.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:27.058+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:27.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:27.058+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:28.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:27 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:27.817+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:28.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:27 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:27.817+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:28.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:27 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:27.817+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:28.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:28 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:28.079+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:28.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:28 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:28.079+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:28.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:28 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:28.079+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:29.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:28 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:28.799+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:29.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:28 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:28.799+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:29.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:28 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:28.799+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:29.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:29 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:29.130+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:29.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:29 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:29.130+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:29.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:29 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:29.130+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:30.270 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:29 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:29.787+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:30.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:29 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:29.787+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:30.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:29 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:29.787+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:30.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:30 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:30.096+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:30.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:30 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:30.096+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:30.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:30 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:30.096+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:31.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:30.737+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:56.241193+0000 front 2024-09-10T23:30:56.241521+0000 (oldest deadline 2024-09-10T23:31:59.741339+0000) 2024-09-10T23:32:31.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:30.737+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.643381+0000 front 2024-09-10T23:31:06.643416+0000 (oldest deadline 2024-09-10T23:32:11.943355+0000) 2024-09-10T23:32:31.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:30.737+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.244546+0000 front 2024-09-10T23:31:14.244714+0000 (oldest deadline 2024-09-10T23:32:20.144409+0000) 2024-09-10T23:32:31.021 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:30 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10[59980]: 2024-09-10T23:32:30.737+0000 7f479d200640 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-09-10T23:31:25.445425+0000 front 2024-09-10T23:31:25.445672+0000 (oldest deadline 2024-09-10T23:32:30.145553+0000) 2024-09-10T23:32:31.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:31 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:31.092+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:31.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:31 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:31.092+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:31.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:31 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:31.092+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:31.934 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:31 smithi092 podman[76684]: 2024-09-10 23:32:31.630237947 +0000 UTC m=+5.896192943 container died 2a978d1a292c286762fe9de3c3b33d3080589be559a0e9af795a6436a2215308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:32:32.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:32 smithi092 podman[76684]: 2024-09-10 23:32:32.074639526 +0000 UTC m=+6.340594528 container cleanup 2a978d1a292c286762fe9de3c3b33d3080589be559a0e9af795a6436a2215308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:32.271 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:32 smithi092 bash[76684]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10 2024-09-10T23:32:32.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:32 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:32.066+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:32.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:32 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:32.066+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:32.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:32 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:32.066+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:32.771 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:32 smithi092 podman[76701]: 2024-09-10 23:32:32.274063319 +0000 UTC m=+0.639021472 container remove 2a978d1a292c286762fe9de3c3b33d3080589be559a0e9af795a6436a2215308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:32:33.095 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:32 smithi092 podman[76794]: 2024-09-10 23:32:32.73256964 +0000 UTC m=+0.045112162 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:32:33.095 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:32 smithi092 podman[76794]: 2024-09-10 23:32:32.875464718 +0000 UTC m=+0.188007239 container create 96c976d8a20ec838fa9e281b5e1a85867c7303fccaab8328245a0ca7c05e5140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T23:32:33.095 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:33 smithi092 podman[76794]: 2024-09-10 23:32:33.094648788 +0000 UTC m=+0.407191310 container init 96c976d8a20ec838fa9e281b5e1a85867c7303fccaab8328245a0ca7c05e5140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T23:32:33.353 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:33 smithi092 podman[76794]: 2024-09-10 23:32:33.101446874 +0000 UTC m=+0.413989396 container start 96c976d8a20ec838fa9e281b5e1a85867c7303fccaab8328245a0ca7c05e5140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T23:32:33.354 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:33 smithi092 podman[76794]: 2024-09-10 23:32:33.234024816 +0000 UTC m=+0.546567352 container attach 96c976d8a20ec838fa9e281b5e1a85867c7303fccaab8328245a0ca7c05e5140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T23:32:33.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:33 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:33.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:33.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:33 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:33.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:33.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:33 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:33.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:33.677 INFO:journalctl@ceph.osd.10.smithi092.stdout:Sep 10 23:32:33 smithi092 podman[76794]: 2024-09-10 23:32:33.353663519 +0000 UTC m=+0.666206042 container died 96c976d8a20ec838fa9e281b5e1a85867c7303fccaab8328245a0ca7c05e5140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:32:34.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T23:32:34.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:34.094 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T23:32:34.095 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T23:32:34.095 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.11 2024-09-10T23:32:34.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:34.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:34.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.077+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:34.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 systemd[1]: Stopping Ceph osd.11 for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:34.879 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.516+0000 7efd52600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:34.880 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.516+0000 7efd52600640 -1 osd.11 141 *** Got signal Terminated *** 2024-09-10T23:32:34.880 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:34 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:34.516+0000 7efd52600640 -1 osd.11 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:32:35.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:35 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:35.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:35.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:35 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:35.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:35.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:35 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:35.073+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:36.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:36 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:36.056+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:36.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:36 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:36.056+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:36.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:36 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:36.056+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:37.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.038+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:37.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.038+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:37.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.038+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:38.379 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.992+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:38.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.992+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:38.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:37 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:37.992+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:39.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:39.010+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-09-10T23:30:53.973642+0000 front 2024-09-10T23:30:53.973690+0000 (oldest deadline 2024-09-10T23:31:59.273620+0000) 2024-09-10T23:32:39.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:39.010+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-09-10T23:31:06.875150+0000 front 2024-09-10T23:31:06.875419+0000 (oldest deadline 2024-09-10T23:32:12.774988+0000) 2024-09-10T23:32:39.380 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:39 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T23:32:39.010+0000 7efd49000640 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-09-10T23:31:14.476084+0000 front 2024-09-10T23:31:14.476130+0000 (oldest deadline 2024-09-10T23:32:20.376121+0000) 2024-09-10T23:32:40.848 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:40 smithi145 podman[79134]: 2024-09-10 23:32:40.486133173 +0000 UTC m=+6.100741241 container died d1d6a144db96194988c2bbc6d3304671b09f9f788bc26a214576e18c2e7ed75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T23:32:41.130 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:40 smithi145 podman[79134]: 2024-09-10 23:32:40.96334938 +0000 UTC m=+6.577957453 container cleanup d1d6a144db96194988c2bbc6d3304671b09f9f788bc26a214576e18c2e7ed75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:32:41.130 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:40 smithi145 bash[79134]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11 2024-09-10T23:32:41.629 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:41 smithi145 podman[79151]: 2024-09-10 23:32:41.205206693 +0000 UTC m=+0.704712062 container remove d1d6a144db96194988c2bbc6d3304671b09f9f788bc26a214576e18c2e7ed75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T23:32:42.043 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:41 smithi145 podman[79238]: 2024-09-10 23:32:41.669430069 +0000 UTC m=+0.027317189 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:32:42.043 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:41 smithi145 podman[79238]: 2024-09-10 23:32:41.81585337 +0000 UTC m=+0.173740474 container create 0a15e28a219369ab50cdacb08d16e56242f960599a63e9a07c12fa6871e1f872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T23:32:42.043 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:42 smithi145 podman[79238]: 2024-09-10 23:32:42.043336439 +0000 UTC m=+0.401223557 container init 0a15e28a219369ab50cdacb08d16e56242f960599a63e9a07c12fa6871e1f872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:32:42.317 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:42 smithi145 podman[79238]: 2024-09-10 23:32:42.050202925 +0000 UTC m=+0.408090034 container start 0a15e28a219369ab50cdacb08d16e56242f960599a63e9a07c12fa6871e1f872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:42.317 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:42 smithi145 podman[79238]: 2024-09-10 23:32:42.167741894 +0000 UTC m=+0.525628994 container attach 0a15e28a219369ab50cdacb08d16e56242f960599a63e9a07c12fa6871e1f872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:32:42.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Sep 10 23:32:42 smithi145 podman[79238]: 2024-09-10 23:32:42.317799717 +0000 UTC m=+0.675686836 container died 0a15e28a219369ab50cdacb08d16e56242f960599a63e9a07c12fa6871e1f872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T23:32:42.962 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T23:32:43.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:43.038 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T23:32:43.038 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T23:32:43.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.a 2024-09-10T23:32:43.463 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:43 smithi067 systemd[1]: Stopping Ceph mds.a for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:43.797 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:43 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a[75599]: 2024-09-10T23:32:43.463+0000 7f0a40e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:43.797 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:43 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a[75599]: 2024-09-10T23:32:43.463+0000 7f0a40e00640 -1 mds.a *** got signal Terminated *** 2024-09-10T23:32:47.777 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:47 smithi067 podman[140699]: 2024-09-10 23:32:47.514509155 +0000 UTC m=+4.175483252 container died e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T23:32:48.047 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:47 smithi067 podman[140699]: 2024-09-10 23:32:47.883507313 +0000 UTC m=+4.544481409 container cleanup e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T23:32:48.047 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:47 smithi067 bash[140699]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a 2024-09-10T23:32:48.404 INFO:journalctl@ceph.mds.a.smithi067.stdout:Sep 10 23:32:48 smithi067 podman[140711]: 2024-09-10 23:32:48.143351686 +0000 UTC m=+0.618116766 container remove e464fcc390d99807cb947cc499244069bb89ae1a2f82c69b4b746955eae2930b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-10T23:32:48.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T23:32:48.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:48.543 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T23:32:48.543 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T23:32:48.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.d 2024-09-10T23:32:48.798 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:48 smithi067 systemd[1]: Stopping Ceph mds.d for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:49.297 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:48 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d[74355]: 2024-09-10T23:32:48.932+0000 7f7ba0a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:49.297 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:48 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d[74355]: 2024-09-10T23:32:48.932+0000 7f7ba0a00640 -1 mds.d *** got signal Terminated *** 2024-09-10T23:32:51.132 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:50 smithi067 podman[140817]: 2024-09-10 23:32:50.834292626 +0000 UTC m=+2.005494112 container died e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T23:32:51.498 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:51 smithi067 podman[140817]: 2024-09-10 23:32:51.246047632 +0000 UTC m=+2.417249117 container cleanup e0cac91f92f7cd5ca7f1b7ef6aa8b4cdd83489fb3268c6d64ed0f9434db4c629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T23:32:51.498 INFO:journalctl@ceph.mds.d.smithi067.stdout:Sep 10 23:32:51 smithi067 bash[140817]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-d 2024-09-10T23:32:51.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T23:32:51.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:51.753 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T23:32:51.753 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T23:32:51.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.g 2024-09-10T23:32:52.047 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:51 smithi067 systemd[1]: Stopping Ceph mds.g for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:52.547 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:52 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g[75192]: 2024-09-10T23:32:52.152+0000 7f943a800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:52.547 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:52 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g[75192]: 2024-09-10T23:32:52.152+0000 7f943a800640 -1 mds.g *** got signal Terminated *** 2024-09-10T23:32:56.210 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:55 smithi067 podman[140938]: 2024-09-10 23:32:55.926737877 +0000 UTC m=+3.896041501 container died 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:32:56.543 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:56 smithi067 podman[140938]: 2024-09-10 23:32:56.325157486 +0000 UTC m=+4.294461109 container cleanup 784e4084029badae9df077756860e4add55e0ec2526fa95e4b85c5b0baafc59a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T23:32:56.543 INFO:journalctl@ceph.mds.g.smithi067.stdout:Sep 10 23:32:56 smithi067 bash[140938]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-g 2024-09-10T23:32:56.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T23:32:56.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:32:56.798 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T23:32:56.798 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T23:32:56.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.j 2024-09-10T23:32:57.049 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:56 smithi067 systemd[1]: Stopping Ceph mds.j for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:32:57.547 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:57 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j[74003]: 2024-09-10T23:32:57.181+0000 7fb1f7000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:32:57.547 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:57 smithi067 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j[74003]: 2024-09-10T23:32:57.181+0000 7fb1f7000640 -1 mds.j *** got signal Terminated *** 2024-09-10T23:32:59.797 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:59 smithi067 podman[141060]: 2024-09-10 23:32:59.503682936 +0000 UTC m=+2.433404312 container died 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T23:33:00.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:59 smithi067 podman[141060]: 2024-09-10 23:32:59.911945159 +0000 UTC m=+2.841666523 container cleanup 683154f2e60ca3ebf8a2c75dae15f79f88547e0a3fbc797c9b84034b4b8e4482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:33:00.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Sep 10 23:32:59 smithi067 bash[141060]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-j 2024-09-10T23:33:00.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T23:33:00.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:00.400 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T23:33:00.400 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T23:33:00.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.b 2024-09-10T23:33:00.770 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:00 smithi092 systemd[1]: Stopping Ceph mds.b for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:01.270 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:00 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: 2024-09-10T23:33:00.783+0000 7f942aa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:01.270 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:00 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b[65469]: 2024-09-10T23:33:00.783+0000 7f942aa00640 -1 mds.b *** got signal Terminated *** 2024-09-10T23:33:02.223 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:01 smithi092 podman[76911]: 2024-09-10 23:33:01.908339143 +0000 UTC m=+1.220051207 container died dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T23:33:02.520 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:02 smithi092 podman[76911]: 2024-09-10 23:33:02.337811343 +0000 UTC m=+1.649523407 container cleanup dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T23:33:02.521 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:02 smithi092 bash[76911]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b 2024-09-10T23:33:02.806 INFO:journalctl@ceph.mds.b.smithi092.stdout:Sep 10 23:33:02 smithi092 podman[76922]: 2024-09-10 23:33:02.548790785 +0000 UTC m=+0.628712053 container remove dd372898bc1aecca9b4ae75ad167886070f93c32a58c1525306e843fe82d06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T23:33:02.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T23:33:02.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:02.897 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T23:33:02.897 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T23:33:02.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.e 2024-09-10T23:33:03.271 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:02 smithi092 systemd[1]: Stopping Ceph mds.e for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:03.771 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:03 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e[64857]: 2024-09-10T23:33:03.313+0000 7f1dbc200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:03.771 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:03 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e[64857]: 2024-09-10T23:33:03.313+0000 7f1dbc200640 -1 mds.e *** got signal Terminated *** 2024-09-10T23:33:06.642 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:06 smithi092 podman[77032]: 2024-09-10 23:33:06.32029141 +0000 UTC m=+3.142283207 container died 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T23:33:06.975 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:06 smithi092 podman[77032]: 2024-09-10 23:33:06.765913501 +0000 UTC m=+3.587905298 container cleanup 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T23:33:06.976 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:06 smithi092 bash[77032]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e 2024-09-10T23:33:07.231 INFO:journalctl@ceph.mds.e.smithi092.stdout:Sep 10 23:33:06 smithi092 podman[77045]: 2024-09-10 23:33:06.975248863 +0000 UTC m=+0.640160273 container remove 039bdd436f0deee22cd2adc6aed7a7456e296806733ee094ec6fa2c223936995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T23:33:07.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T23:33:07.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:07.323 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T23:33:07.323 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T23:33:07.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.h 2024-09-10T23:33:07.520 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:07 smithi092 systemd[1]: Stopping Ceph mds.h for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:08.020 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:07 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: 2024-09-10T23:33:07.730+0000 7fd380200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:08.020 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:07 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h[65169]: 2024-09-10T23:33:07.730+0000 7fd380200640 -1 mds.h *** got signal Terminated *** 2024-09-10T23:33:10.230 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:09 smithi092 podman[77151]: 2024-09-10 23:33:09.906791369 +0000 UTC m=+2.277803143 container died 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:33:10.520 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:10 smithi092 podman[77151]: 2024-09-10 23:33:10.319049377 +0000 UTC m=+2.690061155 container cleanup 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T23:33:10.521 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:10 smithi092 bash[77151]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h 2024-09-10T23:33:10.836 INFO:journalctl@ceph.mds.h.smithi092.stdout:Sep 10 23:33:10 smithi092 podman[77163]: 2024-09-10 23:33:10.562097352 +0000 UTC m=+0.647702773 container remove 40f890febfb7a1f032e9ef68cc33cc15cdfc1ef23f83e88edc5525211c8f5332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T23:33:10.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T23:33:10.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:10.924 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T23:33:10.924 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T23:33:10.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.k 2024-09-10T23:33:11.270 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:10 smithi092 systemd[1]: Stopping Ceph mds.k for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:11.770 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: 2024-09-10T23:33:11.333+0000 7f9c41800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:11.771 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:11 smithi092 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k[64551]: 2024-09-10T23:33:11.333+0000 7f9c41800640 -1 mds.k *** got signal Terminated *** 2024-09-10T23:33:14.216 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:13 smithi092 podman[77272]: 2024-09-10 23:33:13.875543839 +0000 UTC m=+2.681543756 container died ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:33:14.521 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:14 smithi092 podman[77272]: 2024-09-10 23:33:14.322570492 +0000 UTC m=+3.128570409 container cleanup ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T23:33:14.521 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:14 smithi092 bash[77272]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k 2024-09-10T23:33:14.819 INFO:journalctl@ceph.mds.k.smithi092.stdout:Sep 10 23:33:14 smithi092 podman[77283]: 2024-09-10 23:33:14.556733689 +0000 UTC m=+0.655263275 container remove ab7287038c30e37ea2d4e41f0b91078837216a059e7c15baaa623720309e9a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T23:33:14.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T23:33:14.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:14.915 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T23:33:14.915 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T23:33:14.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.c 2024-09-10T23:33:15.324 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:15 smithi145 systemd[1]: Stopping Ceph mds.c for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:15.629 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:15 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: 2024-09-10T23:33:15.322+0000 7fefd0c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:15.629 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:15 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c[66380]: 2024-09-10T23:33:15.322+0000 7fefd0c00640 -1 mds.c *** got signal Terminated *** 2024-09-10T23:33:18.213 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:17 smithi145 podman[79351]: 2024-09-10 23:33:17.88643571 +0000 UTC m=+2.673354364 container died 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T23:33:18.587 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:18 smithi145 podman[79351]: 2024-09-10 23:33:18.327765596 +0000 UTC m=+3.114684254 container cleanup 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:33:18.587 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:18 smithi145 bash[79351]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c 2024-09-10T23:33:18.844 INFO:journalctl@ceph.mds.c.smithi145.stdout:Sep 10 23:33:18 smithi145 podman[79363]: 2024-09-10 23:33:18.587210108 +0000 UTC m=+0.692733931 container remove 97b7f6d18f392c2fd1aa39747ffbb99c2bec64bc3773f21927f64eac5bb469c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:33:18.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T23:33:18.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:18.925 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T23:33:18.925 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T23:33:18.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.f 2024-09-10T23:33:19.129 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:19 smithi145 systemd[1]: Stopping Ceph mds.f for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:19.879 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:19 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f[66082]: 2024-09-10T23:33:19.391+0000 7fe6e2e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:19.880 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:19 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f[66082]: 2024-09-10T23:33:19.391+0000 7fe6e2e00640 -1 mds.f *** got signal Terminated *** 2024-09-10T23:33:22.358 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:21 smithi145 podman[79471]: 2024-09-10 23:33:21.887115577 +0000 UTC m=+2.681416415 container died 8c448e44ed9bd537240ea54eb431eec34245bee50a768d94b0815f26a77c4d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T23:33:22.629 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:22 smithi145 podman[79471]: 2024-09-10 23:33:22.472884394 +0000 UTC m=+3.267185231 container cleanup 8c448e44ed9bd537240ea54eb431eec34245bee50a768d94b0815f26a77c4d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:33:22.630 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:22 smithi145 bash[79471]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f 2024-09-10T23:33:22.993 INFO:journalctl@ceph.mds.f.smithi145.stdout:Sep 10 23:33:22 smithi145 podman[79491]: 2024-09-10 23:33:22.732912695 +0000 UTC m=+0.832418344 container remove 8c448e44ed9bd537240ea54eb431eec34245bee50a768d94b0815f26a77c4d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:33:23.015 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T23:33:23.087 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:23.087 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T23:33:23.087 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T23:33:23.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.i 2024-09-10T23:33:23.379 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:23 smithi145 systemd[1]: Stopping Ceph mds.i for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:23.879 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:23 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: 2024-09-10T23:33:23.538+0000 7f1025600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:23.880 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:23 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i[66679]: 2024-09-10T23:33:23.538+0000 7f1025600640 -1 mds.i *** got signal Terminated *** 2024-09-10T23:33:26.130 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:25 smithi145 podman[79598]: 2024-09-10 23:33:25.869740842 +0000 UTC m=+2.494681543 container died 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:33:26.536 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:26 smithi145 podman[79598]: 2024-09-10 23:33:26.327385198 +0000 UTC m=+2.952325904 container cleanup 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T23:33:26.537 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:26 smithi145 bash[79598]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i 2024-09-10T23:33:26.793 INFO:journalctl@ceph.mds.i.smithi145.stdout:Sep 10 23:33:26 smithi145 podman[79610]: 2024-09-10 23:33:26.536435281 +0000 UTC m=+0.647947215 container remove 20dc6ab074ecde03ba26d4a44f23e158b4bf66c84bf737869c3ced951fac61e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:33:26.814 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T23:33:26.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:26.883 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T23:33:26.884 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T23:33:26.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.l 2024-09-10T23:33:27.129 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:26 smithi145 systemd[1]: Stopping Ceph mds.l for 5c4a3ee4-6fc6-11ef-bcea-c7b262605968... 2024-09-10T23:33:27.630 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l[65770]: 2024-09-10T23:33:27.291+0000 7fed83600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:33:27.630 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:27 smithi145 ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l[65770]: 2024-09-10T23:33:27.291+0000 7fed83600640 -1 mds.l *** got signal Terminated *** 2024-09-10T23:33:31.616 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:31 smithi145 podman[79720]: 2024-09-10 23:33:31.321067753 +0000 UTC m=+4.173690988 container died 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:33:31.879 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:31 smithi145 podman[79720]: 2024-09-10 23:33:31.730183785 +0000 UTC m=+4.582807019 container cleanup 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T23:33:31.879 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:31 smithi145 bash[79720]: ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l 2024-09-10T23:33:32.343 INFO:journalctl@ceph.mds.l.smithi145.stdout:Sep 10 23:33:32 smithi145 podman[79732]: 2024-09-10 23:33:32.047889578 +0000 UTC m=+0.712817664 container remove 7667c2a28a8003b4ccb0e9288e563997499e880bbeb5f8c48c41f17dc7542cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T23:33:32.368 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c4a3ee4-6fc6-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T23:33:32.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:33:32.437 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T23:33:32.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:33:32.721 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:33:36.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:33:36.993 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:33:40.930 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:33:41.202 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:33:45.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:33:45.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:33:45.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:33:45.246 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T23:33:45.249 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi067/crash 2024-09-10T23:33:45.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:33:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:33:45.314 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:33:45.318 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi092/crash 2024-09-10T23:33:45.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:33:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:33:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:33:45.397 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi145/crash 2024-09-10T23:33:45.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:33:45.474 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:33:45.474 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:33:45.476 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T23:33:45.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T23:33:45.632 INFO:tasks.cephadm:Compressing logs... 2024-09-10T23:33:45.633 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:33:45.637 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:33:45.641 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:33:45.700 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T23:33:45.700 INFO:teuthology.orchestra.run.smithi067.stderr:: No such file or directory 2024-09-10T23:33:45.708 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T23:33:45.708 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-09-10T23:33:45.708 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T23:33:45.709 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T23:33:45.709 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T23:33:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T23:33:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:33:45.711 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:33:45.711 INFO:teuthology.orchestra.run.smithi145.stderr:: No such file or directory 2024-09-10T23:33:45.711 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:33:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:33:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T23:33:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T23:33:45.714 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T23:33:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:33:45.715 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T23:33:45.717 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T23:33:45.718 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:33:45.718 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T23:33:45.718 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T23:33:45.718 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T23:33:45.718 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:33:45.720 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T23:33:45.721 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T23:33:45.721 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T23:33:45.722 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T23:33:45.723 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:33:45.725 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T23:33:45.725 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.3.log: 87.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:33:45.725 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:33:45.725 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 95.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T23:33:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.4% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T23:33:45.726 INFO:teuthology.orchestra.run.smithi067.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T23:33:45.727 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T23:33:45.728 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.6% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T23:33:45.729 INFO:teuthology.orchestra.run.smithi145.stderr: 94.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T23:33:45.730 INFO:teuthology.orchestra.run.smithi092.stderr: 94.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T23:33:45.733 INFO:teuthology.orchestra.run.smithi067.stderr: 95.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T23:33:45.735 INFO:teuthology.orchestra.run.smithi067.stderr: 95.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T23:33:45.744 INFO:teuthology.orchestra.run.smithi145.stderr: 82.7% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi067.stderr: 84.7% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:33:45.817 INFO:teuthology.orchestra.run.smithi092.stderr: 82.7% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:33:45.824 INFO:teuthology.orchestra.run.smithi092.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:33:45.842 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:33:45.849 INFO:teuthology.orchestra.run.smithi092.stderr: 90.3% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:33:45.860 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:33:45.898 INFO:teuthology.orchestra.run.smithi145.stderr: 87.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:33:45.908 INFO:teuthology.orchestra.run.smithi145.stderr: 90.4% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:33:45.962 INFO:teuthology.orchestra.run.smithi145.stderr: 90.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:33:46.404 INFO:teuthology.orchestra.run.smithi067.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:33:46.737 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:33:46.844 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:33:46.853 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T23:33:47.279 INFO:teuthology.orchestra.run.smithi145.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:33:47.377 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T23:33:48.865 INFO:teuthology.orchestra.run.smithi145.stderr: 91.7% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T23:33:48.874 INFO:teuthology.orchestra.run.smithi145.stderr: 94.3% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T23:33:49.633 INFO:teuthology.orchestra.run.smithi092.stderr: 91.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T23:33:54.192 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T23:34:01.097 INFO:teuthology.orchestra.run.smithi092.stderr: 94.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T23:34:07.813 INFO:teuthology.orchestra.run.smithi092.stderr: 94.5% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T23:34:13.567 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T23:34:44.577 INFO:teuthology.orchestra.run.smithi092.stderr: 94.2% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T23:38:58.031 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T23:40:12.050 INFO:teuthology.orchestra.run.smithi145.stderr: 92.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T23:40:20.524 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T23:41:06.067 INFO:teuthology.orchestra.run.smithi145.stderr: 92.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T23:41:49.412 INFO:teuthology.orchestra.run.smithi092.stderr: 92.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T23:42:28.862 INFO:teuthology.orchestra.run.smithi092.stderr: 92.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T23:42:28.865 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-10T23:42:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:real 8m43.191s 2024-09-10T23:42:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:user 5m47.044s 2024-09-10T23:42:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m39.288s 2024-09-10T23:42:58.045 INFO:teuthology.orchestra.run.smithi145.stderr: 92.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T23:43:44.730 INFO:teuthology.orchestra.run.smithi145.stderr: 91.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T23:43:44.733 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-10T23:43:44.733 INFO:teuthology.orchestra.run.smithi145.stderr:real 9m59.059s 2024-09-10T23:43:44.733 INFO:teuthology.orchestra.run.smithi145.stderr:user 5m59.935s 2024-09-10T23:43:44.734 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m41.209s 2024-09-10T23:45:28.915 INFO:teuthology.orchestra.run.smithi067.stderr: 92.0% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T23:47:09.559 INFO:teuthology.orchestra.run.smithi067.stderr: 91.9% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T23:50:10.326 INFO:teuthology.orchestra.run.smithi067.stderr: 91.8% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T23:53:34.851 INFO:teuthology.orchestra.run.smithi067.stderr: 91.7% -- replaced with /var/log/ceph/5c4a3ee4-6fc6-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T23:54:54.080 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T23:54:54.083 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-09-10T23:54:54.083 INFO:teuthology.orchestra.run.smithi067.stderr:real 21m8.416s 2024-09-10T23:54:54.083 INFO:teuthology.orchestra.run.smithi067.stderr:user 9m40.363s 2024-09-10T23:54:54.083 INFO:teuthology.orchestra.run.smithi067.stderr:sys 1m18.564s 2024-09-10T23:54:54.084 INFO:tasks.cephadm:Archiving logs... 2024-09-10T23:54:54.085 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi067/log 2024-09-10T23:54:54.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:56:13.588 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi092/log 2024-09-10T23:56:13.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:57:09.824 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi145/log 2024-09-10T23:57:09.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:58:06.633 INFO:tasks.cephadm:Removing cluster... 2024-09-10T23:58:06.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force 2024-09-10T23:58:07.286 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:58:09.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force 2024-09-10T23:58:09.586 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:58:11.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 --force 2024-09-10T23:58:11.767 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 5c4a3ee4-6fc6-11ef-bcea-c7b262605968 2024-09-10T23:58:13.632 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T23:58:13.632 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:58:13.685 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:58:13.725 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:58:13.760 INFO:tasks.cephadm:Teardown complete 2024-09-10T23:58:13.760 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T23:58:13.770 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T23:58:13.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:58:13.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:58:13.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:58:14.017 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T23:58:14.027 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T23:58:14.028 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:58:14.033 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:58:14.037 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:58:14.064 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:58:14.068 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:58:14.071 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -461us[ -458us] +/- 63ms 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 +533us[ +536us] +/- 69ms 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -23us[ -20us] +/- 78ms 2024-09-10T23:58:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -455us[ -455us] +/- 63ms 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +526us[ +526us] +/- 69ms 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -31us[ -31us] +/- 78ms 2024-09-10T23:58:14.190 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -461us[ -458us] +/- 63ms 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +540us[ +543us] +/- 69ms 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -12us[-9465ns] +/- 78ms 2024-09-10T23:58:14.210 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:58:14.213 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T23:58:14.223 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T23:58:14.223 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T23:58:14.232 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:58:14.481 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:58:14.647 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:58:14.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:58:14.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:14.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:58:14.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:14.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:58:14.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:14.986 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T23:58:15.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T23:58:15.448 INFO:teuthology.task.internal:Duration was 4774.258001 seconds 2024-09-10T23:58:15.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T23:58:15.488 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T23:58:15.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:58:15.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:58:15.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:58:15.605 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:58:15.614 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:58:15.630 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:58:16.113 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T23:58:16.113 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-09-10T23:58:16.113 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:58:16.172 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-10T23:58:16.172 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:58:16.226 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T23:58:16.226 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:58:16.283 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T23:58:16.283 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:58:16.287 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:58:16.291 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:58:16.366 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T23:58:16.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:58:16.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:58:16.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:58:17.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T23:58:17.391 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T23:58:17.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:58:17.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:58:17.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:58:17.580 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T23:58:17.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.672 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-09-10T23:58:17.677 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-10T23:58:17.688 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-10T23:58:17.759 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:17.793 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:17.826 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:58:17.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:58:17.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T23:58:17.871 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T23:58:17.873 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi067 2024-09-10T23:58:17.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:58:17.963 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi092 2024-09-10T23:58:17.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:58:18.032 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899748/remote/smithi145 2024-09-10T23:58:18.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:58:18.098 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T23:58:18.098 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:58:18.102 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:58:18.107 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:58:18.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T23:58:18.154 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T23:58:18.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T23:58:18.165 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T23:58:18.165 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:58:18.178 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:58:18.182 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:58:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 393935 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:58 /home/ubuntu/cephtest 2024-09-10T23:58:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: 393245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:58 /home/ubuntu/cephtest 2024-09-10T23:58:18.217 INFO:teuthology.orchestra.run.smithi145.stdout: 393255 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:58 /home/ubuntu/cephtest 2024-09-10T23:58:18.219 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T23:58:18.229 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T23:58:18.254 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 4774.2580008506775 flavor: default mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.b-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.c-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a owner: scheduled_hyelloji@teuthology success: true 2024-09-10T23:58:18.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:58:18.350 INFO:teuthology.run:pass